IS428 AY2019-20T1 Assign Lim Pei Xuan: Data
MC2 will contain two data files spanning the entire length of the simulation (12 am on April 6, 2020 to 11:59 pm on April 10, 2020), containing radiation measurements from mobile and static radiation sensors. MC2 also provides a set of supporting files (described below).
MobileSensorReadings.csv
Contains readings from 50 mobile sensors that are attached to cars. Data fields include: Timestamp, Sensor-id, Long, Lat, Value, Units, User-id. The timestamps are reported in 5 second intervals, though poor data connectivity can result in missing data. Each sensor has a unique identifier that is a number from 1 to 50. Location of the sensor is reported as longitude and latitude values (see map description below). The radiation measurement is provided in the Value field. Radiation is reported with units of counts per minute (cpm). Each measurement is independent and does not represent a summation over the previous minute. Some users have chosen to attach a user ID to their measurements while some others chose with a default name.
Be prepared for missing and corrupted data, skipped timesteps, and other issues. Both radiation measurements and movements may be affected by conditions in the city.
Supporting files
The locations of the static sensors can be found in the file StaticSensorLocations.csv
Several maps have been provided as images, some with labels and some without.
A map of the neighborhoods has also been provided as a shapefile, which is contained in the folder ‘StHimarkNeighborhoodShapefile’. Geometry of the polygons is reported in meters.
Data Preparation
The mobile and static sensor data, along with the locations of each sensor have to be analysed together in order to achieve meaningful analysis of radiation readings. The data preparation was entirely done in Tableau Prep Builder. The final flow is shown below.
| Step | Description |
|---|---|
|
1 |
 Sensor-ids repeat between the mobile and static sensors. To differentiate them, make a new calculated field that labels all mobile sensor data as "Mobile". The New Sensor-id contains the prefix "M" to signify that this is a mobile sensor. The "Units" field is not required as the measurements(cpm) are standard between the static and mobile data. |
|
2 |
 To ensure that the static and mobile sensor data can be combined, the coordinates of the static sensors have to be present. A full outer join between the static sensor readings and static sensor locations was done. |
|
3 |
 Sensor-ids repeat between the mobile and static sensors. To differentiate them, make a new calculated field that labels all static sensor data as "Static". The New Sensor-id contains the prefix "S" to signify that this is a static sensor. The "Units" field is not required as the measurements(cpm) are standard between the static and mobile data. |
|
4 |
Now that the data has the same format and columns, we can union the static and mobile sensor data. |
|
5 |
 The final data file is as above. The data is output as an extract to optimise performance when imported into Tableau. |
Importing data into Tableau
| Step | Description |
|---|---|
|
1 |
In order to be able to aggregate data by neighbourhood, we will have to make use of the shape file provided. To join the data with the polygons from the shape file, we can use a formula to assign the coordinates of the sensor to a specific polygon(neighbourhood). |
