Difference between revisions of "ISSS608 2016-17 T3 Assign Chan En Ying Grace Methodology"

From Visual Analytics and Applications
Jump to navigation Jump to search
Line 86: Line 86:
 
<b>Data Understanding</b>  
 
<b>Data Understanding</b>  
 
||
 
||
<b>1. Read in Raster Layer (Lekagul Roadways Map)</b>  
+
<b>i. Read in Raster Layer (Lekagul Roadways Map)</b>  
 
* It is a single layer raster file. 200x200.  
 
* It is a single layer raster file. 200x200.  
  
Line 98: Line 98:
  
  
<b>2. Find out structure of Raster Layer</b>
+
<b>ii. Find out structure of Raster Layer</b>
 
<br> Extent          : 40000
 
<br> Extent          : 40000
 
<br> CRS arguments  : NA  
 
<br> CRS arguments  : NA  
Line 111: Line 111:
 
<b>Data Cleaning</b>  
 
<b>Data Cleaning</b>  
 
||
 
||
<b>1. Import two CSV Files (Birds)</b>  
+
<b>i. Import two CSV Files (Birds)</b>  
 
* 2081 Training Birds (Metadata)
 
* 2081 Training Birds (Metadata)
 
* 15 Test Birds (Provided by Kasios)
 
* 15 Test Birds (Provided by Kasios)
  
  
<b>2. Fix Data Quality Issues</b>  
+
<b>ii. Fix Data Quality Issues</b>  
 
* Change File ID from numeric to character  
 
* Change File ID from numeric to character  
 
* Change coordinates to numeric
 
* Change coordinates to numeric
Line 128: Line 128:
  
  
<b>3. Data Manipulation</b>  
+
<b>iii. Data Manipulation</b>  
 
* Extract out the “Year” and “Month” from the date, as new columns
 
* Extract out the “Year” and “Month” from the date, as new columns
 
* Create a new column for Quarter (Q1,Q2,Q3,Q4) & Season (Spring, Summer, Fall, Winter)
 
* Create a new column for Quarter (Q1,Q2,Q3,Q4) & Season (Spring, Summer, Fall, Winter)
  
  
<b>4. Geospatial File Compatibility</b>  
+
<b>iv. Geospatial File Compatibility</b>  
 
* Convert CSV file (2081 birds) into the following:
 
* Convert CSV file (2081 birds) into the following:
 
** spatial point data frame  
 
** spatial point data frame  
Line 143: Line 143:
  
  
<b>5. Data Overview & Exploration</b>  
+
<b>v. Data Overview & Exploration</b>  
 
* Overlay 2081 Birds, Raster Map & Dumping Site, for an integrated overview using `plot()`
 
* Overlay 2081 Birds, Raster Map & Dumping Site, for an integrated overview using `plot()`
 
* Use `facet_wrap` to identify location of clustering across species, across time, and across season, and by call/song
 
* Use `facet_wrap` to identify location of clustering across species, across time, and across season, and by call/song
  
  
<b>6. Segregation of Treatment & Control Groups</b>  
+
<b>vi. Segregation of Treatment & Control Groups</b>  
 
* Use ‘Rose Pipits’ as Treatment Group
 
* Use ‘Rose Pipits’ as Treatment Group
 
* Use ‘Ordinary Snape’ and ‘Lesse Birchbeere’ as Control Groups
 
* Use ‘Ordinary Snape’ and ‘Lesse Birchbeere’ as Control Groups
Line 159: Line 159:
 
<b>Geospatial Visualisation </b>  
 
<b>Geospatial Visualisation </b>  
 
||
 
||
<b>Spatial Point Pattern Visualisation (Density-Based Measure) </b>  
+
<b><u>Spatial Point Pattern Visualisation (Density-Based Measure) </u></b>  
  
<b>1. Prepare polygon layer </b>
+
<b>i. Prepare polygon layer </b>
 
* Create a 200x200 spatial polygon to depict the boundaries of Lekagul raster map
 
* Create a 200x200 spatial polygon to depict the boundaries of Lekagul raster map
 
* Merge Raster Polygon with Rose Pipit Layer, using `owin` from spatstat package
 
* Merge Raster Polygon with Rose Pipit Layer, using `owin` from spatstat package
  
<b>2. Kernel Density Plot </b>
+
 
 +
<b>ii. Kernel Density Plot </b>
 
* First, set sigma=bw.diggle  
 
* First, set sigma=bw.diggle  
 
* Apply the Kernel Density Plot (By Year; 2012-2017)
 
* Apply the Kernel Density Plot (By Year; 2012-2017)
Line 172: Line 173:
 
** For OS & LB only (Control Groups)
 
** For OS & LB only (Control Groups)
  
<b>3. Adjust Parameters (sigma) </b>
+
 
 +
<b>iii. Adjust Parameters (sigma) </b>
 
* Adjust the plots by using the sigma of the most dense cluster  
 
* Adjust the plots by using the sigma of the most dense cluster  
 
** This is typically the largest sigma
 
** This is typically the largest sigma
  
 +
 +
<b>iv. Fine-Tune for Clearer Visualisation </b>
 +
* Then add in the dumping site & adjust the colour/size
 +
* So that we can visualize the clusters relative to the dumping site
 +
|-
 +
|
 +
4.
 +
||
 +
<b>Statistical Confirmation </b>
 +
||
 +
<b><u>Spatial Point Pattern Analysis (Distance-Based Measure)</u></b>
 +
 +
<b>i. Quadrat Analysis  </b>
 +
* Apply Monti-Carlo Simulation
 +
* Followed by Quadrat Test to test for clustering
 +
 +
 +
<b>ii. K-Nearest Neighbour  </b>
 +
* Apply Monti-Carlo Simulation
 +
* Followed by Clark-Evans Test to test for clustering
 +
 +
 +
<b>iii. K-Function  </b>
 +
* Apply Monti-Carlo simulation
 +
* Visualise significance based on grey band
 
|}
 
|}
 
</div>
 
</div>
 
<br>
 
<br>

Revision as of 21:18, 22 June 2018

Rose Pipits.png “Mine dear rose pipits, whence did do thou vanish?”

Background

Methodology

Did Rose Pipit kicketh the bucket?

Which song belongs to thee?

Conclusion

 


Tools

R is the primary tool used in this analysis. The following lists the packages used for the project’s scope - for data cleaning, data visualisation, geospatial analysis and audio processing.

  • R libraries
    • sp
    • rgdal
    • sf
    • raster
    • spatstat
    • maptools
    • gplots
    • ggplot2
    • ggmap
    • rasterVis
    • lattice
    • latticeExtra
    • tidyverse
    • zoo
    • tmap
    • reshape2
    • quantmod
    • ggTimeSeries
    • viridis
    • rlang
    • soundgen
    • tuneR
    • phonTools
    • seewave


Approach Taken

The following outlines the approach used for the analysis.

Step

Approach

Description

1.

Data Understanding

i. Read in Raster Layer (Lekagul Roadways Map)

  • It is a single layer raster file. 200x200.

class : RasterLayer
dimensions : 200, 200, 40000 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0, 200, 0, 200 (xmin, xmax, ymin, ymax)
coord. ref. : NA
names : Lekagul_Roadways_2018
values : 0, 255 (min, max)


ii. Find out structure of Raster Layer
Extent : 40000
CRS arguments : NA
File Size : 41078
Object Size : 14376 bytes
Layer : 1

2.

Data Cleaning

i. Import two CSV Files (Birds)

  • 2081 Training Birds (Metadata)
  • 15 Test Birds (Provided by Kasios)


ii. Fix Data Quality Issues

  • Change File ID from numeric to character
  • Change coordinates to numeric
  • Change Date from Character to Date
  • Omit the two NA values for the Y coordinate.
  • Clean the Dates (All standardise to m/d/y. For missing month/year, I will replace with NA. For missing day, I will impute as 1st day of the month.)
  • Clean the Timing (Standardise all to 24 hour formatting. Use “.” instead of ":")
  • Clean the Vocalisation Type (Standardise all to lower case. For values consisting of both ‘song and call’, change to ‘call’, assumed as a sign of distress while ‘song’ is assumed as the default)
  • Clean the Quality (Recode ‘no score’ as ‘NA’)


iii. Data Manipulation

  • Extract out the “Year” and “Month” from the date, as new columns
  • Create a new column for Quarter (Q1,Q2,Q3,Q4) & Season (Spring, Summer, Fall, Winter)


iv. Geospatial File Compatibility

  • Convert CSV file (2081 birds) into the following:
    • spatial point data frame
    • sp format
    • shp format
    • st_read compatible format
    • readOGR compatible format
    • ppp format (for spatstat compatibility)


v. Data Overview & Exploration

  • Overlay 2081 Birds, Raster Map & Dumping Site, for an integrated overview using `plot()`
  • Use `facet_wrap` to identify location of clustering across species, across time, and across season, and by call/song


vi. Segregation of Treatment & Control Groups

  • Use ‘Rose Pipits’ as Treatment Group
  • Use ‘Ordinary Snape’ and ‘Lesse Birchbeere’ as Control Groups
  • Use ‘All Birds’ as third control

3.

Geospatial Visualisation

Spatial Point Pattern Visualisation (Density-Based Measure)

i. Prepare polygon layer

  • Create a 200x200 spatial polygon to depict the boundaries of Lekagul raster map
  • Merge Raster Polygon with Rose Pipit Layer, using `owin` from spatstat package


ii. Kernel Density Plot

  • First, set sigma=bw.diggle
  • Apply the Kernel Density Plot (By Year; 2012-2017)
    • For All Birds
    • For Rose Pipits only (Treatment Group)
    • For OS & LB only (Control Groups)


iii. Adjust Parameters (sigma)

  • Adjust the plots by using the sigma of the most dense cluster
    • This is typically the largest sigma


iv. Fine-Tune for Clearer Visualisation

  • Then add in the dumping site & adjust the colour/size
  • So that we can visualize the clusters relative to the dumping site

4.

Statistical Confirmation

Spatial Point Pattern Analysis (Distance-Based Measure)

i. Quadrat Analysis

  • Apply Monti-Carlo Simulation
  • Followed by Quadrat Test to test for clustering


ii. K-Nearest Neighbour

  • Apply Monti-Carlo Simulation
  • Followed by Clark-Evans Test to test for clustering


iii. K-Function

  • Apply Monti-Carlo simulation
  • Visualise significance based on grey band