Difference between revisions of "ISSS608 2017-18 T3 Assign Liu Yuanjing Methodology"
Yjliu.2017 (talk | contribs) |
Yjliu.2017 (talk | contribs) |
||
(6 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
<!----------- Main Header ------------> | <!----------- Main Header ------------> | ||
− | <div style=background:# | + | <div style=background:#F9EFE9;> |
[[Image:Water pollution.jpg|250px]] | [[Image:Water pollution.jpg|250px]] | ||
<font size = 5; color="#751102"><span style="font-family:Segoe UI Semibold";> Investigating chemical contamination in the Boonsong Lekagul waterways</span> | <font size = 5; color="#751102"><span style="font-family:Segoe UI Semibold";> Investigating chemical contamination in the Boonsong Lekagul waterways</span> | ||
</font></div> | </font></div> | ||
− | + | {|style="background-color:#F9EFE9;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0"; text-align:center;" | | |
− | {|style="background-color:# | ||
| style="font-family:Segoe UI Light; text-align:center;" width="15%" | | | style="font-family:Segoe UI Light; text-align:center;" width="15%" | | ||
Line 42: | Line 41: | ||
==<font size="5"><font color="#000000">'''Methodology '''</font></font>== | ==<font size="5"><font color="#000000">'''Methodology '''</font></font>== | ||
− | <div style="margin:2.5px; padding: 3px; background: # | + | <div style="margin:2.5px; padding: 3px; background: #F9EFE9; font-family: Arial; border-radius: 1px; text-align:left"> |
{| class="wikitable" style="background-color:#FFFFFF;" width="100%" | {| class="wikitable" style="background-color:#FFFFFF;" width="100%" | ||
|- | |- | ||
Line 118: | Line 117: | ||
==<font size="5"><font color="#000000">'''APPROACH TAKEN '''</font></font>== | ==<font size="5"><font color="#000000">'''APPROACH TAKEN '''</font></font>== | ||
− | <div style="margin:0px; padding: 2px; background: # | + | <div style="margin:0px; padding: 2px; background: #F9EFE9; font-family: Arial; border-radius: 1px; text-align:left"> |
{| class="wikitable" style="background-color:#FFFFFF;" width="100%" | {| class="wikitable" style="background-color:#FFFFFF;" width="100%" | ||
|- | |- | ||
Line 152: | Line 151: | ||
As we know , the beginning of a river, when it flows quickly with lots of energy, is called a young river. The river here is smaller and usually has a rapid, tumbling flow that cuts a narrow channel through rocky hills or mountains. And in the map, the above side has many thin water flow and then coverage in southeast direction. Therefore, we can conclude that the waterflow direction from northeast to southeast, where also can determine that the terrain is high in the northwest and low in the southeast. | As we know , the beginning of a river, when it flows quickly with lots of energy, is called a young river. The river here is smaller and usually has a rapid, tumbling flow that cuts a narrow channel through rocky hills or mountains. And in the map, the above side has many thin water flow and then coverage in southeast direction. Therefore, we can conclude that the waterflow direction from northeast to southeast, where also can determine that the terrain is high in the northwest and low in the southeast. | ||
+ | |- | ||
+ | | | ||
+ | |- | ||
+ | | | ||
+ | 2. | ||
+ | || | ||
+ | <b>Data Cleaning </b> | ||
+ | || | ||
+ | <b>i.Import two CSV Files: Boonsong Lekagul waterways readings.csv ;Chemical measure character.csv </b> | ||
+ | |||
+ | <b>ii.Join the two csv files by measures</b> | ||
+ | <b>ii.Exclude “NILL” variables</b> | ||
|- | |- | ||
− | | | + | | |
+ | 3. | ||
+ | || | ||
+ | <b>Geospatial Visualisation </b> | ||
+ | || | ||
+ | <b>i.Generate coordinate for location in the map</b> | ||
+ | * Import the map picture in tableau; | ||
+ | * Simulation a coordinate for locations and record the x-coordinate and y-coordinate in excel. | ||
+ | * Simulation coordinate by water flows based on flowing locations, where could both show waterflow(path) and location in one map | ||
+ | * Combine the simulation x-coordinate and y-coordinate data with Boonsong Lekagul waterways readings by location. | ||
+ | |||
+ | |- | ||
+ | | | ||
+ | |||
+ | |||
+ | 4. | ||
+ | || | ||
+ | <b>prepare tableau analysis sheets </b> | ||
+ | || | ||
+ | <b>Sheets LIST</b> | ||
+ | * Path&Location; | ||
+ | * Water temperature | ||
+ | * Trend of Z-score | ||
+ | * Location Animation | ||
+ | * Macrozoobenthos Analysis | ||
+ | * Location&Measure | ||
+ | * Measure change | ||
+ | |||
|} | |} | ||
+ | |||
+ | <!-- END OF Methodology--> | ||
+ | |||
+ | |||
+ | ==Dashboard Design== | ||
+ | View the interactive Tableau design here: Vast 2018 mini challenge 2 --Like a Duck to water [https://public.tableau.com/profile/yuanjing.liu#!/vizhome/VAST2018Mini-Challenge2LikeaDucktoWater/likeaducktowater/] | ||
+ | |||
+ | <br> | ||
+ | 1.<b>Introduction</b> -- Provide basic information about the story and show the major measure water paths and locations, and also provide number of measure records in past 19 years. | ||
+ | <p>[[File:LYJ-INTRODUCTION.png|900px|center]] </p> | ||
+ | <b>Design Dashboard </b> | ||
+ | |||
+ | 2.<b>Territorial Water Overview</b> --To visualize water temperature change and discount chemical measures by different locations | ||
+ | <p>[[File:LYJ-Overview.png|900px|center]] </p> | ||
+ | |||
+ | 3.<b>Path Analysis</b> -- To visualize the most volatility measures in different paths | ||
+ | <p>[[File:LYJ-PATHS ANALYSIS.png|900px|center]] </p> | ||
+ | |||
+ | |||
+ | 4.<b>Location Analysis</b> --To visualize the most volatility measures in different locations | ||
+ | <p>[[File:LYJ-location analysis.png|900px|center]] </p> | ||
+ | to check the measure changes trend in certain location | ||
+ | <p>[[File:Chai measures changes.gif|900px|center]] </p> | ||
+ | |||
+ | 5. <b>Location Animation Analysis</b> --To discover unusual change in location. | ||
+ | <p>[[File:LYJ-anamation analysis.png|900px|center]] </p> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | <b>v.Location Animation Analysis </b> | ||
+ | To discover unusual change in location. | ||
+ | [[Image:LYJ-anamation analysis.png|900px|center]] |
Latest revision as of 21:26, 8 July 2018
Tools
|
Methodology
Methodology |
Description |
|
Standardization method: |
Because we want to go through all measures in different waterways or locations but measures have different units and if we want see the volatility changes in different paths or location and then identify potential chemical contamination. Therefore, we decided to use Z-Score to modify the data bias. Z-score represents the distance of the original data from the mean, and the standard of the distance measure is the standard deviation.
If the amount of statistical data is sufficient, the Z-score data distribution is satisfied, 68% of the data is distributed between "-1" and "1", and 95% of the data is distributed between "-2" and "2", 99%. The data is distributed between "-3" and "3". You can use this to verify your data. See the Z-score data distribution below: and then we edit the formula in tableau and add one measure variable, called “Z-Score”, the formula as below:
| |
Animation analysis by Z-scores |
In the dataset, there are more than 100 measures in different locations. Then when user to explore measures, it would kill lots of time to identify one by one. Therefore, Z-score could be utilized in making a preliminary screening among so many measures. In the data set, if measures z-score (based on its value) more than 1 or less than -1 would be classify “YES”, and if between [-1.1], would be labeled “NO”. The formula in tableau as below:
| |
Path Analysis |
Some of sampling sites are in the same water ways and part of them in the upper course, some in the middle course, and also have two sampling sites at lower course. And same waterflow (Path) could tell us the whole picture about chemical measures changes. Because the water would flow down along the path and the chemical contaminant also would flow down but a vital importance point needed to be consider is that the consist of chemical measures is always changing and many chemical substances would be diluted, as more tributaries along the path flow into the main stream. Therefore, it’s truly necessary for us to analysis from water path angle to digest the whole picture.
| |
Location & Measures Analysis |
Based on principle of prudence, we also need focus on every measures and invest it one by one to find the more accurately animation pattern. Because different chemical measures increase or decrease have uncertain positive or negative effects to water environment, so we cannot simply assume that the measures increase would cause negative effect to water environment. In this way, we analysis from location with detail value changes by every measure through the select functions.
|
APPROACH TAKEN
Step |
Approach |
Description |
1. |
Data Understanding |
i. Basic Infromation
ii. Map The location of the potential contamination site identified in last year's VAST Challenge. After on-site inspection, Mistford College investigators have most recently found no indication of contamination at that site. And in the map only high the location but didn’t give water flow ways and other information;
The source of the river is where the magic begins. It all starts when rain falls on a mountain top, or when a snowy peak starts to melts. It might simply begin where water springs up through the ground, from the rock beneath. As we know , the beginning of a river, when it flows quickly with lots of energy, is called a young river. The river here is smaller and usually has a rapid, tumbling flow that cuts a narrow channel through rocky hills or mountains. And in the map, the above side has many thin water flow and then coverage in southeast direction. Therefore, we can conclude that the waterflow direction from northeast to southeast, where also can determine that the terrain is high in the northwest and low in the southeast. |
2. |
Data Cleaning |
i.Import two CSV Files: Boonsong Lekagul waterways readings.csv ;Chemical measure character.csv ii.Join the two csv files by measures ii.Exclude “NILL” variables
|
3. |
Geospatial Visualisation |
i.Generate coordinate for location in the map
|
|
prepare tableau analysis sheets |
Sheets LIST
|
Dashboard Design
View the interactive Tableau design here: Vast 2018 mini challenge 2 --Like a Duck to water [1]
1.Introduction -- Provide basic information about the story and show the major measure water paths and locations, and also provide number of measure records in past 19 years.
Design Dashboard
2.Territorial Water Overview --To visualize water temperature change and discount chemical measures by different locations
3.Path Analysis -- To visualize the most volatility measures in different paths
4.Location Analysis --To visualize the most volatility measures in different locations
to check the measure changes trend in certain location
5. Location Animation Analysis --To discover unusual change in location.
v.Location Animation Analysis
To discover unusual change in location.