Difference between revisions of "IS428 AY2019-20T1 Assign Greg Tan Boon Kit Data Analysis & Transformation"
(Created page with "== Data Analysis & Transformation== The primary step to every data visualisation would be to always ensure that the data is processed and ready for proper data visualisation....") |
|||
(2 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
+ | <b><font size = 6; color="#43464b"> VAST Challenge 2019 Mini-Challenge 1:<br>Crowdsourcing for Situational Awareness</font></b> | ||
+ | </div> | ||
+ | <!--MAIN HEADER --> | ||
+ | {|style="background-color:#aec6cf;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0" | | ||
+ | | style="font-family:Century Gothic; font-size:100%; solid #000000; background:##aec6cf; text-align:center;" width="25%" | | ||
+ | ; | ||
+ | [[IS428 AY2019-20T1 Assign Greg Tan Boon Kit|<b><font size="3"><font color="#43464b">Problem Description</font></font></b>]] | ||
+ | |||
+ | | style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:##aec6cf; text-align:center;" width="25%" | | ||
+ | ; | ||
+ | [[IS428 AY2019-20T1 Assign Greg Tan Boon Kit_Data Analysis & Transformation|<b><font size="3"><font color="#43464b">Data Analysis & Transformation</font></font></b>]] | ||
+ | |||
+ | | style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:##aec6cf; text-align:center;" width="25%" | | ||
+ | ; | ||
+ | [[IS428 AY2019-20T1 Assign Greg Tan Boon Kit_Data Visualisation|<b><font size="3"><font color="#43464b">Data Visualisation</font></font></b>]] | ||
+ | |||
+ | | style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:##aec6cf; text-align:center;" width="25%" | | ||
+ | ; | ||
+ | [[IS428 AY2019-20T1 Assign Greg Tan Boon Kit_Task Questions & Answers| <b><font size="3"><font color="#43464b">Task Questions & Answers</font></font></b>]] | ||
+ | |||
+ | | | ||
+ | |} | ||
+ | |||
+ | |||
+ | |||
== Data Analysis & Transformation== | == Data Analysis & Transformation== | ||
The primary step to every data visualisation would be to always ensure that the data is processed and ready for proper data visualisation. | The primary step to every data visualisation would be to always ensure that the data is processed and ready for proper data visualisation. | ||
Line 35: | Line 60: | ||
# Load Tableau Prep and connect data source (mc1-reports-data.csv) | # Load Tableau Prep and connect data source (mc1-reports-data.csv) | ||
# Click on 'Pivot' from the '+' icon and select the relevant variables | # Click on 'Pivot' from the '+' icon and select the relevant variables | ||
+ | |} | ||
+ | |||
+ | [[File:Instrumental Intensity.png|frameless|800px|center]] | ||
+ | {| class="wikitable" | ||
+ | |- | ||
+ | ! style="background:#B0E0E6; |Issues Faced | ||
+ | |- | ||
+ | | | ||
+ | '''Issue''': The damage done to the various infrastructure is in a numerical format. At first glance, it might not be clear to emergency responders based on a numerical scale which area is badly damaged and needs immediate assistance. | ||
+ | |- | ||
+ | | | ||
+ | '''Solution''': Making use of Tableau Prep to create a calculated field, the damage done to the various infrastructure can be tagged by their Instrumental Intensity. | ||
+ | |||
+ | This would make it easier for the emergency respondents to identify at once which areas are the worst hit based on the existing Instrumental Intensity indicator. | ||
+ | To solve this issue: | ||
+ | # Click on 'Clean' from the '+' icon | ||
+ | # Set the conditions to match each numerical number to the respective category as seen in the image above | ||
+ | |} | ||
+ | |||
+ | [[File:Processed Data.png|frameless|800px|center]] | ||
+ | After processing, the data is now prepared and ready for visualisation. The processed data can be then exported into a separate output.csv file. | ||
+ | |||
+ | [[File:mapping.png|frameless|800px|center]] | ||
+ | {| class="wikitable" | ||
+ | |- | ||
+ | ! style="background:#B0E0E6; |Issues Faced | ||
+ | |- | ||
+ | | | ||
+ | '''Issue''': After loading the processed csv file into Tableau and loading the StHimark.shp file, there is a disjoint in the data in terms of how the two files are linked. | ||
+ | |- | ||
+ | | | ||
+ | '''Solution''': Use inner join when connecting the two files to ensure that the data is correctly linked between the files to ensure that proper data visualisation can take place subsequently. | ||
+ | |||
+ | To solve this issue: | ||
+ | # Click on 'Add' under Connections in Data Source | ||
+ | # Choose the StHimark.shp file to upload to the tableau workbook | ||
+ | # Choose 'Inner Join' | ||
+ | # Under 'Data Source', choose 'location' and under 'StHimark.shp', choose 'Id' | ||
+ | |||
+ | The above step serves to connect the location, which is a number representing each neighbourhood with Id, which is also the number in the StHimark.shp file that uniquely identifies each neighbourhood. | ||
|} | |} |
Latest revision as of 01:29, 13 October 2019
VAST Challenge 2019 Mini-Challenge 1:
Crowdsourcing for Situational Awareness
|
|
|
|
Data Analysis & Transformation
The primary step to every data visualisation would be to always ensure that the data is processed and ready for proper data visualisation.
In this case, most of the data was collected from citizens of St. Himark through the use of the Rumble App. Capitalising on the benefit of crowd sourced information from the citizens ensure that emergency responders know which area is most disaster stricken, and can target those areas first.
Data collected from the Rumble includes both Categorical and Continuous data, with the main data collected being: 1. Timestamps of reports from citizens 2. The intensity of the earthquake shaking at a particular timestamp 0: Lowest, 10: Highest 3. The different types of infrastructure (eg. Buildings, Power, Sewers, Medical, Roads) 4. The ID of each neighbourhood’s location 5. Damage done to each type of infrastructure from the earthquake 0: Not Felt, 10: Extreme
Issues Faced |
---|
Issue: Data provided with regards to the details of various facilities are in their own specific rows. This poses a challenge for further visualisation as each data is not being properly segmented. |
Solution: Making use of Tableau Prep, the columns for the various infrastructure's details can be pivoted to show just 1 column, labelled infrastructure. This column would include the data from previous columns:
This would streamline all the data under each of this infrastructure type to ensure proper data visualisation can be done. To solve this issue:
|
Issues Faced |
---|
Issue: The damage done to the various infrastructure is in a numerical format. At first glance, it might not be clear to emergency responders based on a numerical scale which area is badly damaged and needs immediate assistance. |
Solution: Making use of Tableau Prep to create a calculated field, the damage done to the various infrastructure can be tagged by their Instrumental Intensity. This would make it easier for the emergency respondents to identify at once which areas are the worst hit based on the existing Instrumental Intensity indicator. To solve this issue:
|
After processing, the data is now prepared and ready for visualisation. The processed data can be then exported into a separate output.csv file.
Issues Faced |
---|
Issue: After loading the processed csv file into Tableau and loading the StHimark.shp file, there is a disjoint in the data in terms of how the two files are linked. |
Solution: Use inner join when connecting the two files to ensure that the data is correctly linked between the files to ensure that proper data visualisation can take place subsequently. To solve this issue:
The above step serves to connect the location, which is a number representing each neighbourhood with Id, which is also the number in the StHimark.shp file that uniquely identifies each neighbourhood. |