Difference between revisions of "IS428 AY2019-20T1 Assign Nurul Khairina Binte Abdul Kadir Data"
Line 36: | Line 36: | ||
== Data Description == | == Data Description == | ||
− | The first step in the transformation process begins with understanding and interpreting the data to determine which data type we currently have and what we need to transform it into. | + | The first step in the transformation process begins with understanding and interpreting the data to determine which data type we currently have and what we need to transform it into. |
− | The data zip file consists of 1 CSV file (mc1-reports-data.csv) spanning the entire length of the event from 6 April 2020, 12 AM to 11 April 2020, 12 AM and 2 shakemap images. The CSV file contains the individual reports (categorical) of shaking/damage by neighborhood over time. Reports are made by citizens at any time | + | The data zip file consists of 1 CSV file (mc1-reports-data.csv) spanning the entire length of the event from 6 April 2020, 12 AM to 11 April 2020, 12 AM and 2 shakemap images. The CSV file contains the individual reports (categorical) of shaking/damage by neighborhood over time. Reports are made by citizens at any time through the Rumble mobile app. However, they are only recorded in 5-minute batches/increments due to the server configuration. Furthermore, delays in the receipt of reports may occur during power outages. |
'''Data Attributes of CSV File''' | '''Data Attributes of CSV File''' |
Revision as of 01:51, 9 October 2019
VAST 2019 MC1: Crowdsourcing for Situational Awareness
|
|
|
|
|
Contents
Data Description
The first step in the transformation process begins with understanding and interpreting the data to determine which data type we currently have and what we need to transform it into.
The data zip file consists of 1 CSV file (mc1-reports-data.csv) spanning the entire length of the event from 6 April 2020, 12 AM to 11 April 2020, 12 AM and 2 shakemap images. The CSV file contains the individual reports (categorical) of shaking/damage by neighborhood over time. Reports are made by citizens at any time through the Rumble mobile app. However, they are only recorded in 5-minute batches/increments due to the server configuration. Furthermore, delays in the receipt of reports may occur during power outages.
Data Attributes of CSV File
Data Attributes | Description |
---|---|
Time | Timestamp of incoming report/record, in the format YYYY-MM-DD hh:mm:ss |
Location | ID of neighborhood where person reporting is feeling the shaking and/or seeing the damage |
Shake Intensity, Sewer and Water, Power, Roads and Bridges, Medical, Buildings | Reported categorical value of how violent the shaking was/how bad the damage was (0 - lowest, 10 - highest; missing data allowed) |
Shakemap Images, Shape File and Map Images
Also included are two shakemap (PNG) files which indicate where the corresponding earthquakes' epicenters originate as well as how much shaking can be felt across the city. The StHimarkNeighbourhoodShapefile and map images which are obtained from the dataset of Mini Challenge 2 is used to create the map of the city.
Dataset Analysis & Transformation Process
The following section illustrates the issues faced in the data analysis phase leading to a need to transform the data into a specified format. Tableau Prep will be used to clean and prepare the data for analysis.
Data Manipulation for MC1-Report-Data CSV File
There are 3 main issues with the given dataset and data transformation is needed to reshape the data for easier analysis.
Pivot the Damage Types
Issue 1: Inability to filter by damage type
Solution: Pivot the following columns: Shake Intensity, Sewer and Water, Power, Roads and Bridges, Medical and Buildings. By pivoting the data, we can create more rows for each
By pivoting the data we can create rows for each month and year and individual columns for sales, taxes, and totals so that Tableau can more easily interpret this data for analysis.