Difference between revisions of "IS428 AY2019-20T1 Assign Chye Soon Hang Data Cleaning"
Jump to navigation
Jump to search
VAST Challenge 2019 MC 2: Managing Radiation Together As A Nation
Shchye.2017 (talk | contribs) (Created page with "<div style=background:#f4c20d border:#fff> <center><font size = 5; color="#fff"><b>VAST Challenge 2019 MC 2: Managing Radiation Together As A Nation</center></b></font> </div...") |
Shchye.2017 (talk | contribs) |
||
(One intermediate revision by the same user not shown) | |||
Line 30: | Line 30: | ||
<p>This dashboard is supported by the following 4 datasets, given in CSV and SHP formats:</p> | <p>This dashboard is supported by the following 4 datasets, given in CSV and SHP formats:</p> | ||
<ul> | <ul> | ||
− | <li>StaticSensorLocations.csv – contains locations of the static sensors</li | + | <li>StaticSensorLocations.csv – contains locations of the static sensors</li> |
<li>StaticSensorReadings.csv – contains readings from 9 static sensors that are situated either around hospitals if not somewhere near to Always Safe Nuclear Power Plant</li> | <li>StaticSensorReadings.csv – contains readings from 9 static sensors that are situated either around hospitals if not somewhere near to Always Safe Nuclear Power Plant</li> | ||
<li>MobileSensorReadings.csv - contains readings from 50 mobile sensors that are attached to cars</li> | <li>MobileSensorReadings.csv - contains readings from 50 mobile sensors that are attached to cars</li> | ||
Line 41: | Line 41: | ||
<li>Join StaticSensorLocations.csv and StaticSensorReadings.csv via the Join function</li> | <li>Join StaticSensorLocations.csv and StaticSensorReadings.csv via the Join function</li> | ||
<ul> | <ul> | ||
− | <li>Set Join type to be Inner Join and Join Clause to be matching Sensor-id</li> | + | <li>Set Join type to be Inner Join and Join Clause to be matching Sensor-id<br> |
+ | [[File:JoinStatic.png|300px]] | ||
+ | </li> | ||
<li>This will help to give each static sensor record its respective locations (Long, Lat)</li> | <li>This will help to give each static sensor record its respective locations (Long, Lat)</li> | ||
− | <li>Due to conflicting sensor ids with mobile sensors, I have change all the static sensor id to start from 60 to 68.</li> | + | <li>Due to conflicting sensor ids with mobile sensors, I have change all the static sensor id to start from 60 to 68.<br> |
+ | [[File:StaticSensorID.png|200px]]</li> | ||
</ul> | </ul> | ||
+ | <li>Merge Static Sensors Records together with Mobile Sensors Records via the Union Function<br> | ||
+ | [[File:CSVMerge.png|270px]] | ||
+ | </li> | ||
+ | <li>Export combined dataset via Output function | ||
+ | <ul> | ||
+ | <li>Set the export version to .hyper so that the accessing speed will be much faster compared to .csv. | ||
+ | <br> | ||
+ | [[File:PrepOutput.png|270px]] | ||
+ | </li> | ||
+ | </ul> | ||
+ | </li> | ||
+ | </ol> | ||
+ | |||
+ | <b>Final Workflow in Tableau Prep</b><br> | ||
+ | [[File:Workflow.png|600px]] | ||
+ | |||
+ | <h2>Importing Combined Dataset and StHimark.shp into Tableau Desktop</h2> | ||
+ | <ol> | ||
+ | <li>Join the CombinedDataset with StHimark.shp | ||
+ | <ul> | ||
+ | <li>Via Left Join Function as we want to keep the unmatched records in CombinedDataset (Static Records)</li> | ||
+ | <li>Join Clause will be MAKEPOINT([Lat], [Long]) from CombinedDataset intersecting with Geometry from StHimark.shp | ||
+ | <br> | ||
+ | [[File:MakePoint.png|270px]] | ||
+ | <br> | ||
+ | [[File:LeftJoin.png|320px]] | ||
+ | </li> | ||
+ | </ul> | ||
+ | </li> | ||
+ | <li>Set connection to be Extract for better loading speed</li> | ||
</ol> | </ol> |
Latest revision as of 12:49, 13 October 2019
Data Cleaning
This dashboard is supported by the following 4 datasets, given in CSV and SHP formats:
- StaticSensorLocations.csv – contains locations of the static sensors
- StaticSensorReadings.csv – contains readings from 9 static sensors that are situated either around hospitals if not somewhere near to Always Safe Nuclear Power Plant
- MobileSensorReadings.csv - contains readings from 50 mobile sensors that are attached to cars
- StHimark.shp – contains geometry and sensor id for individual neighborhood
Data Transformation on Tableau Prep
- Import all CSV files on to Tableau Prep
- Join StaticSensorLocations.csv and StaticSensorReadings.csv via the Join function
- Set Join type to be Inner Join and Join Clause to be matching Sensor-id
- This will help to give each static sensor record its respective locations (Long, Lat)
- Due to conflicting sensor ids with mobile sensors, I have change all the static sensor id to start from 60 to 68.
- Merge Static Sensors Records together with Mobile Sensors Records via the Union Function
- Export combined dataset via Output function
Final Workflow in Tableau Prep
Importing Combined Dataset and StHimark.shp into Tableau Desktop
- Join the CombinedDataset with StHimark.shp
- Set connection to be Extract for better loading speed