IS428 AY2018-19T1 Chen Yuge

Problem & Motivation

As one of the most polluted countries in Europe, Bulgaria is facing a high level of pollution. It is ranked eighth in the European Environment Agency’s 2017 report on air quality in Europe in terms of most premature deaths caused by PM2.5[1]. Our goal in this report is to find the pollution condition in the capital of Bulgaria– Sofia city.

In this report, I will use 3 Air Quality indicators—Official Air Quality Concentration, PM2.5 and PM10 to analyze the air quality in Sofia city. PM2.5 is a pollutant stemming from fuel combustion, heating, transportation, waste incineration, agriculture and other anthropogenic sources. According to studies, it is highly correlated with cancer rate, with a 36% increase in lung cancer per 10 μg/m3 as it can penetrate deeper into the lungs[2]. Worldwide exposure to PM2.5 contributed to 4.1 million deaths from heart disease and stroke, lung cancer, chronic lung disease, and respiratory infections. PM10 is particulate matter 10 micrometers or less in diameter, which has slight larger dimension than PM2.5 but same level of danger.

By using the above-mentioned indicators, the goal is to visualize the air quality situation as well as the factors affecting the air quality. The factors which will be analyzed in this report are:

     -	Local energy sources

     -	Meteorology such as temperature, pressure, rainfall, humidity, wind etc

     -	Human Behavior such as driving, room heating

     -	Behaviors of Neighbors of Sofia city.

Data Analysis & Transformation

Data Analysis and Cleaning

Air Tube data

After doing a quick plotting of the raw data, I found that there are outliers (or misreading) in the Air Tube Dataset: Temperature: remove temperature less than -20 and larger than 50

Column Name	Action	Screenshot
Geocoding	Transform geohash to latitude and longtitude
Temperature	remove temperature less than -20 and larger than 50
Pressure	remove pressures valued 0

Aggregation

EEA data

• Aggregate data from different years (2013-2018) into one sheet with “day” details:
  

Steps

1. Aggregate average concentration by day in each data sheets from 2013-2018

1. Combine sheets of different stations and years into 1 (2013-2018)

• Aggregate 2018 dataset into one sheet with “hour” details:

Steps

1. Transform 2018 data time to local time & take average time

   * As I explored the dataset, I found the datetime format is UCT. I changed it to local time.
   * The time interval between "DatetimeBegin" column and "Datetime End" is always 1 hour, therefore I decided to take the middle time--0.5 hour late than "DatetimeBegin" as the time of the record.

1. Aggregate 5 sheets

Visualization

Tableau Link: unable to upload to tableau online because the data amount exceeds the limit.

Home Page

Button	Description	icon
Official Air Quality Station Timeseries	Timeseries line chart of Average concentration across 5 stations. map of location of 5 stations as well as it's average concentration across years (2013-2017) Dataset: EEA data
Official Air Quality Heatmap	Timeseries heatmap of: Concentration by weekdays across years (2013-2018) Concentration in 24 hours across years month animation (2018) Concentration in 24 hours across years overview (2018) Dataset: EEA data
Sensor Distribution Map	Timeseries line chart of Average concentration across 5 stations. map of location of 5 stations as well as it's average concentration across years (2013-2017) Dataset: EEA data
Factors Analysis	Timeseries line chart of Average concentration across 5 stations. map of location of 5 stations as well as it's average concentration across years (2013-2017) Dataset: EEA data

=== Official Air Quality ===

2013-2018

Feature	Description
Flat layout of concentration across years	The 6 line charts put in parallel allows user to compare the data between years more easily.
Highlight across 6 line charts	All lines of same station will be highlighted when click on one line. This provide user with more intuitive visualization of change across years within same location.
Animation by month on map of stations	The orange color dots represents the location of stations, and the darker the color is, the higher concentration the location is. The animation allows user to see the change of stations locations and concentration value across time in different years. This gives user an intuitive overview of the air quality change in recent years.

2018

Feature	Description
Weekdays breakdown of concentration	Considering weekdays as one of the potential influencing factors on air pollution, I drawed a headmap of concentration distribution by weekdays across years (2013-2017) to find our the partterns.
Hourly breakdown of cencentration	As part of incluencing factors, I drew the concentration timeseries breakdown by 24 hours.
heatmap animation by month	Heatmap animaiton show the data transition between months more clearly

Trends:

1. Concentration is higher during night and morning time, lower at noon. This may be due to the traffic pollution during peak hours.
2. The concentration is almost identical across different weekdays. This means weekdays is not a huge influencing factor on air quality.
3. Generally speaking, Concentration is becoming lower from year of 2013 to today. This means the air pollution problem is being solved.

Anomalies

• Some of the data missing such as 2017 Jan to Oct. Might because of breakdown of devices
  
• The concentration in January and February is significantly higher than other months in each year.

===Citizen Science Air Quality===

Sensor geographical Distribution

• Across Cities: Sensors are evenly spread out across cities except Sofia city and Polvdiv, which has more condensed sensor distribution.
  

• Inside Sofia City: The sensors are more condensed in the middle of city. It is not evenly distributed in the entire city.

Sensing data statistics

Feature	Description
animation on weekly distribution of PM2.5 & PM10 value on map	The animations shows intuitively the data distribution across years
Boxplot of PM2.5 & PM10 value across weeks	Box plot shows not only the average, but also the spread of value, which helps user to identify the outliers and understand data distribution better.

Observation:

1. Week 1, 2 and 4 (1st, 2nd and 4th week of January 2018) has very high concentration, whereas the concentration in week 2 and the rest weeks are low.
2. The high concentration points are located at the north part of Sofia city
3. In boxplot week5, there is a high outlier of concentration. This means something abnormal happened in that week.

Factors influencing air quality

• Scatter plot

We can deduce the following information from the scatter plot:

1. PM2.5 and PM10 are higher when pressure is around 1000k and 170k
2. PM2.5 and PM10 are positive correlated. it’s possible to have high PM10 and low PM2.5, but when PM10 is low, PM2.5 must below
3. PM2.5 and PM10 are higher when temperature is around 0 slowly reduce as temperature increase. It reaches near to 0 when temperature is close to 40
4. The upper bond of PM2.5 & PM10 increases as humidity increase. But there is no linear relationship between humidity and PM2.5&10.
   

Reference

[1] https://zerowasteeurope.eu/2018/01/bulgaria-air-pollution/
[2] https://en.wikipedia.org/wiki/Particulates

Comments

Feel free to leave your comments here