Difference between revisions of "IS428 AY2018-19T1 Kung Jung-wen"
Jwkung.2015 (talk | contribs) |
Jwkung.2015 (talk | contribs) |
||
Line 12: | Line 12: | ||
2. All the CSV files provided contains the air quality station identifiers, air pollution measurement, time period of measurement and other links. To obtain the geographic location of each air station, we then merged the data with metadata using left merge in Python to obtain mainly the longitude and latitude points of each station. | 2. All the CSV files provided contains the air quality station identifiers, air pollution measurement, time period of measurement and other links. To obtain the geographic location of each air station, we then merged the data with metadata using left merge in Python to obtain mainly the longitude and latitude points of each station. | ||
− | 3. | + | |
− | change date variable to a format that tableau can read. | + | 3. The date variables need to be standardised and cleaned up in a format that is readable in Tableau. |
+ | change date variable to a format that tableau can read. The current DatetimeBegin and DatetimeEnd variables contain time in milliseconds, as the data will only be analysed on either an hourly or daily level, the date of transformed to include the date and hour of the measurement time in Python. | ||
[[File:EEA date transform.jpg|600px|center|EEA date transform.jpg]] | [[File:EEA date transform.jpg|600px|center|EEA date transform.jpg]] | ||
− | + | ||
− | |||
− | |||
− | |||
− | |||
− | |||
== Dataset Import Structure & Process == | == Dataset Import Structure & Process == | ||
Line 62: | Line 58: | ||
Using the dashboard as a platform for investigation and analysis, the following aims to provide answers to the questions posed. | Using the dashboard as a platform for investigation and analysis, the following aims to provide answers to the questions posed. | ||
− | === Q1: | + | === Q1: Spatio-temporal Analysis of Official Air Quality === |
− | ''' | + | '''Characteristic of the past and most recent situation with respect to air quality measures in Sofia City'''<br/> |
+ | [[File:Overview of EEA.png|200px|frame|center]] | ||
+ | # The overall trend of slight improvement in air quality: Sofia City’s PM10 level is on the fall, air quality is becoming better over the years with lower peaks in 2015 onwards compared to 2014. The PM10 level reached a new minimum level in 2018 with a PM 10 concentration of 12.2. | ||
+ | # Overall improvement in air quality in 2018: from the chart showing the PM10 Concentration Fluctuation by Month, every month in 2018 has the lowest PM10 concentration compared to previous years. | ||
+ | |||
+ | '''Anomaly pattern in PM10 Concentration''' | ||
+ | |||
+ | # Extremely poor daily PM10 Concentration in December and January: from Daily PM10 Concentration Chart, most of the peaks are around December and January period and the PM10 level can reach as high as 400+ range. | ||
+ | # Poor monthly average PM10 concentration in 2014 and 2016: The two peaks shown in the Monthly Avg PM10 Concentration Chart, are December 2014 and January 2016 which is consistent with the findings above. | ||
+ | # No improvement in monthly PM10 concentration for January and November: as it was found that the air quality had an overall improvement in 2018, however it seems that the average monthly PM10 level in January and November is on the rise. We cannot make this conclusion about improved air quality in 2018 November and December as the data is not available yet. However, it seems that the main activity that causes of poor air quality is being shifted to November. | ||
+ | |||
+ | |||
Based on the data captured by the proximity sensors, the following shows a typical pattern in the proximity card data. | Based on the data captured by the proximity sensors, the following shows a typical pattern in the proximity card data. | ||
# For the fixed proximity sensors, readings are collected for 24hours during the weekdays, except for 1am and 4am. From 12midnight to 6am, only level 1 fixed proximity sensor will collect data. This can possibly mean that there is no employee movement from 12midnight to 6am on level 2 and 3. | # For the fixed proximity sensors, readings are collected for 24hours during the weekdays, except for 1am and 4am. From 12midnight to 6am, only level 1 fixed proximity sensor will collect data. This can possibly mean that there is no employee movement from 12midnight to 6am on level 2 and 3. |
Revision as of 14:49, 11 November 2018
Contents
Problem & Motivation
Dataset Transformation Process
Before starting with the analysis in Tableau, each feature within the data set is being analysed to better understand the context of the problem and to ensure that the data transformation process is performed accurately.
Working with EEA Data:
1. Merge all the EEA air quality data: As there are 28 csv files being provided from the data source, we first combining all the data within python.
2. All the CSV files provided contains the air quality station identifiers, air pollution measurement, time period of measurement and other links. To obtain the geographic location of each air station, we then merged the data with metadata using left merge in Python to obtain mainly the longitude and latitude points of each station.
3. The date variables need to be standardised and cleaned up in a format that is readable in Tableau. change date variable to a format that tableau can read. The current DatetimeBegin and DatetimeEnd variables contain time in milliseconds, as the data will only be analysed on either an hourly or daily level, the date of transformed to include the date and hour of the measurement time in Python.
Dataset Import Structure & Process
Interactive Visualization
The interactive visualization can be accessed here: https://public.tableau.com/viws/Assignment3_145/Home?:embed=ey&:display_count=yes
Interactive Technique | Rationale | Brief Implementation Steps |
---|---|---|
The use of checkboxes or dropdown list requires the analyst to check/uncheck each date manually which is time-consuming. As such, a time range slider is preferred. |
| |
Use of a drop down list also allow analysts to easily choose the building level that they are interested to analyse. |
| |
When a user filters from one floor to another, the floor plan also changes to provide for quick and easy reference. Due to the space constraint, each floor plan has to be zoomed for users to identify and see the zone areas clearly. |
|
Interesting & Anomalous Observations
Using the dashboard as a platform for investigation and analysis, the following aims to provide answers to the questions posed.
Q1: Spatio-temporal Analysis of Official Air Quality
Characteristic of the past and most recent situation with respect to air quality measures in Sofia City
- The overall trend of slight improvement in air quality: Sofia City’s PM10 level is on the fall, air quality is becoming better over the years with lower peaks in 2015 onwards compared to 2014. The PM10 level reached a new minimum level in 2018 with a PM 10 concentration of 12.2.
- Overall improvement in air quality in 2018: from the chart showing the PM10 Concentration Fluctuation by Month, every month in 2018 has the lowest PM10 concentration compared to previous years.
Anomaly pattern in PM10 Concentration
- Extremely poor daily PM10 Concentration in December and January: from Daily PM10 Concentration Chart, most of the peaks are around December and January period and the PM10 level can reach as high as 400+ range.
- Poor monthly average PM10 concentration in 2014 and 2016: The two peaks shown in the Monthly Avg PM10 Concentration Chart, are December 2014 and January 2016 which is consistent with the findings above.
- No improvement in monthly PM10 concentration for January and November: as it was found that the air quality had an overall improvement in 2018, however it seems that the average monthly PM10 level in January and November is on the rise. We cannot make this conclusion about improved air quality in 2018 November and December as the data is not available yet. However, it seems that the main activity that causes of poor air quality is being shifted to November.
Based on the data captured by the proximity sensors, the following shows a typical pattern in the proximity card data.
- For the fixed proximity sensors, readings are collected for 24hours during the weekdays, except for 1am and 4am. From 12midnight to 6am, only level 1 fixed proximity sensor will collect data. This can possibly mean that there is no employee movement from 12midnight to 6am on level 2 and 3.
- For the mobile proximity sensor, Rosie (the mobile robot) travels the halls at 9am and 2pm daily. On 4, 5, 11 and 12 June, the robot does not collect any readings. One possible reason is that these are weekends (Sat/Sun) and therefore, Rosie will not be required to travel along the hallways.
Typical Day for GAStech Employees
The following lists a typical day for GAStech employees, in all departments:
- Employees start arriving in the office at 7am. By 9am, majority of the employees would have already arrived in the building and settle in their own office space.
- Lunch time is usually between 12nn to 2pm.
- At around 2pm, many employees will be back in their office and that’s when we can see lots of activities going on in the different levels.
- Employees typically end work at around 5pm, though some may leave slightly earlier.
- After 6pm, majority of the employees in the office are from the Engineering, Facilities and IT department.
- After 7pm, employees working in level 3 would have left the office.
- At 12midnight, most of the employees would have left the building and only people in the facilities team will be present in level 1.
- People in the facilities department will be available in the building 24 hours and its always the same people patrolling around the area.
- Other than the facilities personnel, IT and engineering employees often stay in the office and will only leave at around 12midnight.
The following lists some activities that is part of a typical day pattern for employees in a specific department:
Department | Activities |
---|---|
| |
| |
| |
| |
|