Wyz-Methodology
Vast2011 MC1: Characterization of an Epidemic Spread
|
|
|
|
|
Methodology
Text mining
Since the data source is consist of a mass of text messages, it is necessary to extract useful information from it.
Text mining is a process and technic of deriving high-quality information from text. By using the Selection function in JMP, text related to flu can be easily extracted by setting several conditions. After filtering the source data, only 7% of data was related to the topic and 93% of the text were talking about other irrelevant topics.
Another text mining method is to explore the words frequency. In this content, words with high frequency mean that a part of people had same health problems and feelings, which may provide a reliable improvement of infectious disease. What’s more, words frequency can also be used to see influent incidents in specific date or place even the following impact caused by the incident.
Tool: SAS JMP
Data visualization
In data visualization, SAS JMP and Tableau were utilized to present the data. In JMP, the text explorer function can not only find the frequent terms and phrases but also draw word cloud which shows the frequency of each word by size and color. The word with relatively bigger size represents higher frequency. However, it would be difficult for JMP to draw various clear and beautiful plots. So another professional tool, Tableau, was introduced into this part.
Tableau is an industry leading BI tool that focuses on data visualization, dashboarding and data discovery, which compensates the weakness of data processing tools on visualization. With the map and processed location in the given materials and combined with scatter point plot, text information can be visualized geographically. Besides, line charts, bar charts and treemap which are common and useful visualization form were applied for rough exploration.
Tool: SAS JMP, Tableau
