ISSS608 2017-18 T1 Assign XING SIYUAN Data Preparation

From Visual Analytics and Applications
Revision as of 21:44, 15 October 2017 by Siyuan.xing.2016 (talk | contribs)
Jump to navigation Jump to search

Skull.jpg Epidemic Spread in Smartpolis - Origin and Transmission

Background

Data Preparation & Dashboard Design

Insights

Conclusion

 


Data Preparation

Description Illustration
1.Data Cleaning


Tools: JMP
Method:
1. Split Created_at into Data and Time.
2. Split Location into Latitude and Longitude.
3. Exclude 21 rows with invalid Time input.
4. Export table into CSV format.

SY Clean data.png
2.Identify infected patients


Tools: Tableau
By loading the cleaned data into Tableau, we can draw a heat map to visualize the macroblog density per day in each location. From the heatmap of number of macroblogs, we know that there is a huge increase in the number of macroblogs posted on 19th & 20th of May. There must be some major events that caused the increase of macroblogs.
From the locations of macroblogs posted on 19th and 20th of May (as shown in the left figue), it is obvious that there is a high density of macroblogs around the hospital of Smartpolis (highlighted with black square). Which means these posts has a high possibility that is being posted by people who has been infected by the epidemic. By investigate what has those people posted and where has those people been to in the last few day can help us find where the outbreak started, how the infection is being transmitted and measure whether the outbreak is contained or not.
Select the macroblogs on map where the location is around the hospitals. Group user_id of these posts and create a set named patients. Extract a csv file that contains ID of all the people in patients set.

Heatmap of Number of Macroblogs by days:

SY num dis.png

Macroblogs distribution in the last day:

SY patients.png
3.Identify Symptom of Infected Patients


Tools: JMP
Load patients ID file into JMP and join it with Macroblogs table. With text explorer of JMP, the top mentioned words and phrase posted by infected people are generated (left top figure). By filtering the words that are related to symptoms of the epidemic, we can tell that most patients were suffering a fever, cough, headache, diarrhea, vomit, sore throat, aching muscles, runny nose, difficulty in breath and so on.
By further investigating in the symptoms, it seems that the symptoms can be clustered into two categories, one related to gastrointestinal discomfort, the other related to inhalation discomfort. Hence, it is possible that the epidemic contains two type of diseases and may has two origins and multiple transmission methods. We chose 7 words from inhalation symptoms and 4 from gastrointestinal symptoms (shown on the left middle table) to identify origin and transmission method of the epidemic.
Create 11 columns with col_name of the 11 words selected, check if the text in each row contains the corresponding words, if yes, out put 1. If no, output 0. Formula:

SY formula.png






Tools: Tableau

Data integration Text Explorer
SY p m.png
SY words.png

Words table:

Symptom Type Words
Inhalation chill, flu, sore throat, breath, pneumonia, fever, cough
Gastrointestinal stomachache, diarrhea, vomit, nausea
4.Identify Major Events in Smartpolis


5.Overall visualization design concepts


6 types of visitors, 5 categories of gates and 3 time intervals for analysis – this challenge deem to be an intriguing one. Thus, the visualizations were designed make full use of Tableau’s interactivity such that we can provide a tool for the users to customize their analysis as much as possible. The users are able to filter the different input parameters (visitor type, gates, time), choose the extent of animation (for the paths) and choose what is the area of focus for the analysis by playing with the colors. This will be explained in greater details in the dashboard designs section.