IS428 AY2019-20T1 Assign Chye Soon Hang Data Cleaning

VAST Challenge 2019 MC 2: Managing Radiation Together As A Nation

Problem & Motivation

Data Cleaning

Dashboard

Observations

References

Data Cleaning

This dashboard is supported by the following 4 datasets, given in CSV and SHP formats:

• StaticSensorLocations.csv – contains locations of the static sensors
• StaticSensorReadings.csv – contains readings from 9 static sensors that are situated either around hospitals if not somewhere near to Always Safe Nuclear Power Plant
• MobileSensorReadings.csv - contains readings from 50 mobile sensors that are attached to cars
• StHimark.shp – contains geometry and sensor id for individual neighborhood

Data Transformation on Tableau Prep

1. Import all CSV files on to Tableau Prep
2. Join StaticSensorLocations.csv and StaticSensorReadings.csv via the Join function
• Set Join type to be Inner Join and Join Clause to be matching Sensor-id
  
• This will help to give each static sensor record its respective locations (Long, Lat)
• Due to conflicting sensor ids with mobile sensors, I have change all the static sensor id to start from 60 to 68.
  
3. Merge Static Sensors Records together with Mobile Sensors Records via the Union Function
   
4. Export combined dataset via Output function
   • Set the export version to .hyper so that the accessing speed will be much faster compared to .csv.
     

Final Workflow in Tableau Prep

Importing Combined Dataset and StHimark.shp into Tableau Desktop

1. Join the CombinedDataset with StHimark.shp
   • Via Left Join Function as we want to keep the unmatched records in CombinedDataset (Static Records)
   • Join Clause will be MAKEPOINT([Lat], [Long]) from CombinedDataset intersecting with Geometry from StHimark.shp
     
     
2. Set connection to be Extract for better loading speed