IS428 AY2019-20T1 Assign Chye Soon Hang Data Cleaning

From Visual Analytics for Business Intelligence
Revision as of 12:49, 13 October 2019 by Shchye.2017 (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
VAST Challenge 2019 MC 2: Managing Radiation Together As A Nation

Problem & Motivation

 

Data Cleaning

 

Dashboard

 

Observations

 

References

Data Cleaning

This dashboard is supported by the following 4 datasets, given in CSV and SHP formats:

  • StaticSensorLocations.csv – contains locations of the static sensors
  • StaticSensorReadings.csv – contains readings from 9 static sensors that are situated either around hospitals if not somewhere near to Always Safe Nuclear Power Plant
  • MobileSensorReadings.csv - contains readings from 50 mobile sensors that are attached to cars
  • StHimark.shp – contains geometry and sensor id for individual neighborhood

Data Transformation on Tableau Prep

  1. Import all CSV files on to Tableau Prep
  2. Join StaticSensorLocations.csv and StaticSensorReadings.csv via the Join function
    • Set Join type to be Inner Join and Join Clause to be matching Sensor-id
      JoinStatic.png
    • This will help to give each static sensor record its respective locations (Long, Lat)
    • Due to conflicting sensor ids with mobile sensors, I have change all the static sensor id to start from 60 to 68.
      StaticSensorID.png
  3. Merge Static Sensors Records together with Mobile Sensors Records via the Union Function
    CSVMerge.png
  4. Export combined dataset via Output function
    • Set the export version to .hyper so that the accessing speed will be much faster compared to .csv.
      PrepOutput.png

Final Workflow in Tableau Prep
Workflow.png

Importing Combined Dataset and StHimark.shp into Tableau Desktop

  1. Join the CombinedDataset with StHimark.shp
    • Via Left Join Function as we want to keep the unmatched records in CombinedDataset (Static Records)
    • Join Clause will be MAKEPOINT([Lat], [Long]) from CombinedDataset intersecting with Geometry from StHimark.shp
      MakePoint.png
      LeftJoin.png
  2. Set connection to be Extract for better loading speed