Difference between revisions of "Weekly Diary 1"

From Analytics Practicum
Jump to navigation Jump to search
Line 283: Line 283:
 
<!---------------END of sub menu ---------------------->
 
<!---------------END of sub menu ---------------------->
  
<big>Data Preparation</big>
+
Our first meeting in school term. We started by cleaning the client data, separating and extracting only the countries of interest.<br>
<p>As the original data files do not contain column headers, R is used to add column headers to each csv file so as to aid data exploration in the next stage easier. </p>
+
While cleaning we realised that all the files came without headers and the headers were separate. This was an issue as if we opened the files in JMP_Pro we wouldn't be able to read and crawl efficiently. So R was used to add column headers to each csv file so as to aid data exploration in the next stage easier.
 
[[File:RCode.JPG|700px]]
 
[[File:RCode.JPG|700px]]
 +
 +
<br>
 +
Furthermore, there were some issues with missing columns in the data as well. We will clarify them during the next scheduled sponsor meeting.
 +
<br>
 +
In preparation for our geospatial analysis we crawled for Australia and Japan's subzone shp files as well.

Revision as of 14:56, 5 January 2017

Logo Hiryuu.png


Home

About Us

Project Overview

Findings

Project Management

Documentations

Timeline

Tasks Start End Teammates Involved Status
Data Gathering and Scoping
Gather Data Week 0 Week 0 All Completed
Finalise Requirements with client Week 0 Week 0 All Completed
Scope Project Week 0 Week 0 All Completed
Research and Preparation
Explore software Week 0 Week 0 All Completed
Finalise Proposal Week 0 Week 0 All Completed
Create and update wiki page Week 0 Week 0 All Completed
Data Cleaning
Data collection Week 1 Week 2 All Not Completed
Data cleaning and restructuring Week 1 Week 2 All Not Completed
Resolve/remove incomplete data Week 1 Week 2 All Not Completed
Data Modelling
Stage 1: Exploratory Analysis Week 3 Week 6 All Not Completed
Stage 2: Clustering Week 3 Week 6 All Not Completed
Stage 3: Exploratory Analysis Week 3 Week 6 Jouta Not Completed
Stage 4: Geospatial Week 5 Week 10 Qianpin Not Completed
Interim Preparation
Gather feedback from Client Week 4 Week 8 All Not Completed
Prepare interim report and slides Week 6 Week 7 Jouta Not Completed
Application Building
Code the application Week 8 Week 11 All Not Completed
Testing the application Week 10 Week 12 All Not Completed
Gather feedback from Client Week 12 Week 16 All Not Completed
Iteration
Adjust analysis Week 12 Week 13 Jouta Not Completed
Refine results to improve clarity Week 12 Week 13 Wan Theng, Qianpin Not Completed
Final Preparation
Prepare Research Paper Week 14 Week 14 All Not Completed
Prepare Poster Week 15 Week 15 Qianpin Not Completed

Work Plan

  • Fortnightly meetings with Supervisor
  • Monthly meetings with Sponsor
Hiryuu Gantt Chart.JPG

Weekly Diary

Week 0 Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Week 13 Week 14 Week 15 Week 16

Our first meeting in school term. We started by cleaning the client data, separating and extracting only the countries of interest.
While cleaning we realised that all the files came without headers and the headers were separate. This was an issue as if we opened the files in JMP_Pro we wouldn't be able to read and crawl efficiently. So R was used to add column headers to each csv file so as to aid data exploration in the next stage easier. RCode.JPG


Furthermore, there were some issues with missing columns in the data as well. We will clarify them during the next scheduled sponsor meeting.
In preparation for our geospatial analysis we crawled for Australia and Japan's subzone shp files as well.