Development of Automated Environmental Data Collection System and Environment Statistics Dashboard
DOI:
https://doi.org/10.29244/ijsa.v5i2p314-325Keywords:
big data, environment statistics, pollutant, socio-economic, web scrapingAbstract
Environmental data such as pollutants, temperature, and humidity are data that have a role in the agricultural sector in predicting rainfall conditions. In fact, pollutant data is common to be used as a proxy to see the density of industry and transportation. With this need, it is necessary to have automated data from outside websites that are able to provide data faster than satellite confirmation. Data sourced from IQair, can be used as a benchmark or confirmative data for weather and environmental statistics in Indonesia. Data is taken by scraping method on the website. Scraping is done on the API available on the website. Scraping is divided into 2 stages, the first is to determine the location in Indonesia, the second is to collect statistics such as temperature, humidity, and pollutant data (AQI). The module used in python is the scrapy module, where the crawling is effective starting from May 2020. The data is recorded every three hours for all regions of Indonesia and directly displayed by the Power BI-based dashboard. We also illustrated that AQI data can be used as a proxy for socio-economic activity and also as an indicator in monitoring green growth in Indonesia.
Downloads
References
Bogdan, M. (2015). Temperature and Humidity Measurement System.
BPS Statistics Indonesia. (2020a). Big Data Review of The Impact of COVID-19.
BPS Statistics Indonesia. (2020b). Study Big Data as Complete Social Statistic Data and Information.
Chaves A. (2020). Scrapy 2.3 documentation. (Python Software Foundation). Retrieved from https://docs.scrapy.org/en/latest/
Durcevic S. (2020). Utilize The Potential Of Digital Dashboards In A Business Environment. Retrieved from https://www.datapine.com/blog/digital-dashboard-definition-and-examples/#: :text=Digital%20dashboards%20not%20only%20help,discovery%20of%20priceless%20new%20insights
Hananto, V. R., & Putra, I. (2018). A Dashboard System for Monitoring Air Pollution in Surabaya based on PM2. 5. Journal of Information Systems Engineering and Business Intelligence, 4(2): 139–147.
Mawonike, R., & Mandonga, G. (2018). The effect of temperature and relative humidity on rainfall in Gokwe region, Zimbabwe: A factorial design perspective. British View, 3(2).
National laboratory of the U.S. Department of Energy. (2011). Indoor Temperature and Humidity Data Collection and Analysis.
Pramana, S., Paramartha, D. Y., Adhinugroho, Y., & Nurmalasari, M. (2020). Air Pollution Changes of Jakarta, Banten, and West Java, Indonesia During the First Month of COVID-19 Pandemic. The? Journal of Business, Economics, and Environmental Studies (JBEES), 10(4): 15–19.
Pramana, S., Yuniarto, B., Kurniawan, R., Yordani, R., Lee, J., Amin, I., … Indriani, R. (2017). Big data for government policy: Potential implementations of bigdata for official statistics in Indonesia. 2017 International Workshop on Big Data and Information Security (IWBIS), 17–21. IEEE.
