ISSS608 2017-18 T1 Assign XING SIYUAN Data Preparation

Epidemic Spread in Smartpolis - Origin and Transmission

Background

Data Preparation & Dashboard Design

Insights

Conclusion

Data Preparation

Description	Illustration
1.Data Cleaning Tools: JMP Method: 1. Split Created_at into Data and Time. 2. Split Location into Latitude and Longitude. 3. Exclude 21 rows with invalid Time input. 4. Export table into CSV format.
2.Identify infected patients Tools: Tableau By loading the cleaned data into Tableau, we can draw a heat map to visualize the macroblog density per day in each location. From the heatmap of number of macroblogs, we know that there is a huge increase in the number of macroblogs posted on 19th & 20th of May. There must be some major events that caused the increase of macroblogs. From the locations of macroblogs posted on 19th and 20th of May (as shown in the left figue), it is obvious that there is a high density of macroblogs around the hospital of Smartpolis (highlighted with black square). Which means these posts has a high possibility that is being posted by people who has been infected by the epidemic. By investigate what has those people posted and where has those people been to in the last few day can help us find where the outbreak started, how the infection is being transmitted and measure whether the outbreak is contained or not. Select the macroblogs on map where the location is around the hospitals. Group user_id of these posts and create a set named patients. Extract a csv file that contains ID of all the people in patients set.	Heatmap of Number of Macroblogs by days: Macroblogs distribution in the last day:
3.Identify Symptom of Infected Patients - Data Preparation Tools: JMP Load patients ID file into JMP and join it with Macroblogs table. With text explorer of JMP, the top mentioned words and phrase posted by infected people are generated (left top figure). By filtering the words that are related to symptoms of the epidemic, we can tell that most patients were suffering a fever, cough, headache, diarrhea, vomit, sore throat, aching muscles, runny nose, difficulty in breath and so on. By further investigating in the symptoms, it seems that the symptoms can be clustered into two categories, one related to gastrointestinal discomfort, the other related to inhalation discomfort. Hence, it is possible that the epidemic contains two type of diseases and may has two origins and multiple transmission methods. We chose 7 words from inhalation symptoms and 4 from gastrointestinal symptoms (shown on the left middle table) to identify origin and transmission method of the epidemic. Create 11 columns with col_name of the 11 words selected, check if the text in each row contains the corresponding words. If yes, out put 1. If no, output 0.	Words detection: Text Explorer Words table: Symptom Type Words Inhalation chill, flu, sore throat, breath, pneumonia, fever, cough Gastrointestinal stomachache, diarrhea, vomit, nausea Check Symptoms: Formula
4.Identify Origin of the Epidemic - Data Visulazation Tools: Tableau To identify origin of the epidemic, we need to identify both when and where the symptoms start to appeal. First, load the newly created table into tableau and join it with the original data with id and time of each row. To identify start time of the epidemic, we can use line chart to visualize the change in frequency of the symptoms across time (as shown on left). The pink charts represents the frequency of inhalation symptoms and the yellow charts represents the frequency of gastrointestinal symptoms. To identify start location of the epidemic, we can visualize the location of macroblogs that contains the symptoms. To make the visualization more interactive, we can write formula in Tableau to group the symptoms in to inhalation and gastrointestinal. Then add inhalaton and gastrointestinal into filter, so that we can see origin of both type of disease.	Visualization: Appeal Frequency Appeal Location
5.Identify Major Events in Smartpolis	500px