Difference between revisions of "ISS608 2017-18 T1 Assign KyonghwanKim Data Preparation"

From Visual Analytics and Applications
Jump to navigation Jump to search
Line 47: Line 47:
 
*Created_at column is splitted to Date and Time columns. Date column is used in other analytics.<br/>
 
*Created_at column is splitted to Date and Time columns. Date column is used in other analytics.<br/>
 
*Also, Location column is splitted to Latitude and Longitude columns. These data is used to plot in Vastropolis map.
 
*Also, Location column is splitted to Latitude and Longitude columns. These data is used to plot in Vastropolis map.
−
|[[file:microblog_split.png]]
+
|[[file:microblog_split.png|450px]]
 
|-
 
|-
 
|'''2. Outliers'''<br/>
 
|'''2. Outliers'''<br/>
Line 53: Line 53:
 
*Also, there are 6 items with Longitude outside of given map range. They are removed as well so that all data are within parameters.
 
*Also, there are 6 items with Longitude outside of given map range. They are removed as well so that all data are within parameters.
 
*Total 27 rows are removed and 1,023,050 rows are used for analysis with file name "Microblog_Final.csv".
 
*Total 27 rows are removed and 1,023,050 rows are used for analysis with file name "Microblog_Final.csv".
−
|[[file:missing_time.png]]    [[file:outlier_Longitude.png]]
+
|[[file:missing_time.png|150px]]    [[file:outlier_Longitude.png|300px]]
 
|-
 
|-
 
|}
 
|}
  
 
==2. Key Words==
 
==2. Key Words==
−
Following 17 key words and 2 phrases are used for text analysis. Additionally, word "flu" and "cold" are categorized as '''Diagnosis'''. This is because these words actually describe our target status.
+
Some of key words are given. However, there may be additional key words to enhance accuracy of analysis. <br/>
−
<br/>
+
 
 +
[[file:outbreak.png|500px]]
 +
 
 +
Above graph shows text distribution with key word ''"flu"'' and ''"cold"''. Text traffic shoots up from May 18th and remain high until 20th. Text distribution between 3 days of outbreak using JMP text explorer is shown below.
 +
 
 +
[[file:could18.png|250px]][[file:could19.png|250px]][[file:could20.png|250px]]
 +
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 69: Line 75:
 
|-
 
|-
 
|}
 
|}
−
If only '''Diagnosis''' key words are selected from dataset, contagion shoot up since May 18th, 2011 as shown on below graph. According to CDC, ''flu symptoms start 1 to 4 days after the virus enters the body. That means that you may be able to pass on the flu to someone else before you know you are sick, as well as while you are sick.''[https://www.cdc.gov/flu/about/disease/spread.htm] This analysis look closely from 4 days prior to outbreak, which is May 14th, 2011.<br/>
 
−
 
−
[[file:outbreak.png]]
 

Revision as of 05:23, 15 October 2017

Title.png

Vastropolis Epidemic Report

Background

Data Preparation

Visualization

Answer

Reference

Feedback

 



Microblog

1. Data cleaning

Description Illustration
1. Split of Columns
  • Created_at column is splitted to Date and Time columns. Date column is used in other analytics.
  • Also, Location column is splitted to Latitude and Longitude columns. These data is used to plot in Vastropolis map.
 Microblog split.png
2. Outliers
  • There are 21 items with invalid time format. They are removed from analysis.
  • Also, there are 6 items with Longitude outside of given map range. They are removed as well so that all data are within parameters.
  • Total 27 rows are removed and 1,023,050 rows are used for analysis with file name "Microblog_Final.csv".
 Missing time.png Outlier Longitude.png

2. Key Words

Some of key words are given. However, there may be additional key words to enhance accuracy of analysis.

Outbreak.png

Above graph shows text distribution with key word "flu" and "cold". Text traffic shoots up from May 18th and remain high until 20th. Text distribution between 3 days of outbreak using JMP text explorer is shown below.

Could18.pngCould19.pngCould20.png

Diagnosis Symptoms
flu, cold fever, chill, fatigue, cough, breath, nausea, vomit, diarrhea, sweat, pain, sore throat, muscle, letharg (-y or -ic), runny nose, doctor, sick
Retrieved from ‘https://wiki.smu.edu.sg/1718t1isss608g1/index.php?title=ISS608_2017-18_T1_Assign_KyonghwanKim_Data_Preparation&oldid=1422’