Difference between revisions of "ChargeMetrics Proposal"

From Visual Analytics for Business Intelligence
Jump to navigation Jump to search
 
(25 intermediate revisions by the same user not shown)
Line 24: Line 24:
 
<br>
 
<br>
 
<!--/Header-->
 
<!--/Header-->
<!-- START PROJECT MOTIVATION -->
+
<!-- START MOTIVATION -->
  
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px"><span style="font-size:100%;">P</span>roject Motivation</div>==
+
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px"><span style="font-size:100%;">Motivation</div>==
  
<!-- END PROJECT MOTIVATION -->
+
<!-- END MOTIVATION -->
  
===Background===
+
<!-- START OBJECTIVES -->
  
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px"><span style="font-size:100%;">Objectives</div>==
  
===Data===
+
<!-- END OBJECTIVES -->
  
 +
 +
<!-- START DATA -->
 +
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px">Data</div>==
 +
{| class="wikitable" style="background-color:#FFFFFF;" width="100%"
 +
|-
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;width: 30%;" | Dataset/Source
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;width: 30%" | Data Attributes
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;" | Rationale Of Usage
 +
|-
 +
| <center>Resales flat prices from Mar 2012 onwards<br/>
 +
(https://data.gov.sg/dataset/resale-flat-prices?resource_id=83b2fc37-ce8c-4df4-968b-370fd818138b ) </center>
 +
<center>Resales flat prices from 2002 - Feb 2012 <br/>
 +
(https://data.gov.sg/dataset/resale-flat-prices?resource_id=8c00bf08-9124-479e-aeca-7cc411d884c4 ) </center>
 +
<center>Resales flat prices from 1990 - 1999<br/>
 +
(https://data.gov.sg/dataset/resale-flat-prices?resource_id=adbbddd3-30e2-445f-a123-29bee150a6fe ) </center>
 +
||
 +
* Data Type 1
 +
* Data Type 1
 +
||
 +
<center>This dataset will be used as a main source of information in our analysis to understand the number of HDB around Singapore from 1990 to 1999, 2002 to Feb 2012 and Mar 2012 onwards respectively.</center>
 +
|-
 +
| <center>Bus Stop Names and Locations<br/>
 +
(https://www.mytransport.sg/content/mytransport/home/dataMall.html#)</center>
 +
[[File:Bus data.png|center|500px]]
 +
||
 +
* Bus Stop Number
 +
* Bus Stop Roof Number
 +
* Bus Stop Name
 +
* X
 +
* Y
 +
* Latitude
 +
* Longitude
 +
||
 +
<center>This dataset aims to complement the main dataset by providing detailed information about the latitude and longitude of the bus stops located around HDB. We use a javascript geocoding script to convert all the X and Y coordinates to EPSG:4326 latitude and longitude coordinates.</center>
 +
|-
 +
| <center>Mrt Stations Names and Locations <br/>
 +
(https://www.mytransport.sg/content/mytransport/home/dataMall.html#)</center>
 +
[[File:MRT data.png|center|500px]]
 +
||
 +
* MRT Station Number
 +
* MRT Station Name
 +
* X
 +
* Y
 +
* Latitude
 +
* Longitude
 +
||
 +
<center>This dataset aims to complement the main dataset by providing detailed information about the latitude and longitude of the MRT stations located around HDB. We use a javascript geocoding script to convert all the X and Y coordinates to EPSG:4326 latitude and longitude coordinates</center>
 +
|-
 +
| <center>Master Plan Subzone Boundary Names and GeoPolygon <br/>
 +
(https://data.gov.sg/dataset/master-plan-2014-subzone-boundary-no-sea)</center>
 +
[[File:Subzone1.png|center|500px]]<br/>
 +
[[File:Subzone2.png|center|500px]]
 +
||
 +
* Polygon
 +
* Name
 +
* Subzone Number
 +
* Subzone Code
 +
* Region Name
 +
* Area Code
 +
* Area Indicator
 +
||
 +
<center>This dataset aims to complement the main dataset by providing detailed information about all the subzone in Singapore. We use a javascript library toGeoJson.js to help us convert .KML file to .GeoJson file</center>
 +
|}
 +
 +
<!-- END  DATA  -->
 +
 +
<!-- START Related Works -->
 +
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px">Related Works</div>==
 +
 +
<!-- END  Related Works  -->
 +
<p>The following related work </p>
 +
 +
There are many charts and visualisations available which illustrates the various trends of house prices and index. We have selected a few of these to study and learn before we begin developing our own visualizations.
 +
{| class="wikitable" style="background-color:#FFFFFF;" width="100%"
 +
|-
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;width: 50%;" | Related Works
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;" | What We Can Learn
 +
|-
 +
|
 +
<p><center>'''An Analysis of the trend and correlation between resale prices and flat production''' </center></p>
 +
[[File:1718T1G1 BackgroundSurvey1.png|400px|center]]
 +
<p><center>'''Source''': http://www.teoalida.com/singapore/hdbprices/</center></p>
 +
||
 +
* The use of 2 different chart types with a secondary axis is effective in illustrating the correlation between resale prices and flat production.
 +
* The colours used are striking and contrast well with each other.
 +
* There are dips  in both variables which are not explained in the infographic itself (E.g. 1997 Asian crisis, 2003 SARs outbreak). This events could be incorporated into the charts to make it more informative.
 +
* However, major events are not shown in the chart itself. This makes it difficult for the audience to understand the various changes throughout the years. It would be much clearer if events were annotated directly on the charts.
 +
|-
 +
| <p><center> '''An interactive heatmap of Singapore’s house prices in various districts''' </center></p>
 +
[[File:1718T1G1 BackgroundSurvey2.png|400px|center]]
 +
<p><center> '''Source''': https://www.srx.com.sg/heat-map </center> </p>
 +
||
 +
* This heatmap uses colours appropriately so that the house prices of each district can be identified intuitively (Red means expensive, blue means cheap, orange means mid-range)
 +
* The use of filters allows user to find out more about the price distribution of each house type easily.
 +
* When user mouseover a district on the heatmap, the corresponding district on the legend is highlighted. This improves usability as users do not have to match district numbers manually.
 +
* Even though the colours are intuitive, it could still be confusing as audience might still have to constantly refer back to the colour scale to interpret the colours. Colour intensity of a single colour could be used instead of to illustrate the difference in prices.
 +
|-
 +
| <p><center> '''An interactive visualization of house prices along MRT stations''' </center></p>
 +
[[File:1718T1G1 BackgroundSurvey3.png|400px|center]]
 +
<p><center> '''Source''': https://www.srx.com.sg/mrt-home-prices/property-listings-near-east-west-line </center></p>
 +
||
 +
* This visualization makes use of unique ways to illustrate the relation between nearby facilities and house prices. Thus in a glance, a person can tell where are the expensive areas based on the MRT stations.
 +
* The separating of the various MRT lines using filters at the top prevent too much information from being shown in one page
 +
* However, this chart could be too high level and general. Illustrating this on the Singapore map would be more useful instead, where each house's proximity to the MRT station and their respective resale price is shown. Also, in order to drill down, you would have to navigate to another page and then search for a specific house based on a table view which does not help one visualize where the resale flat is actually located at.
 +
|-
 +
| <p><center> '''Four Ways to Slice Obama’s 2013 Budget Proposal''' </center></p>
 +
[[File:4waystosliceobamaproposal.png|400px|center]]
 +
<p><center> '''Source''': http://www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html </center></p>
 +
||
 +
* This visualization makes use of unique ways to illustrate Obama's Budget Proposal
 +
* The various circles denote the various types of proposed budget items and the circles are sized based on the proposed budget amount.
 +
* Users can toggle the links to slice and dice the initial visualization into different views so that a user can get different insights from the same initial visualization.
 +
* This chart is very interesting and it is something that we wanted to use to present the changes in resale prices based on different flat types. If we have the time, we could do different filters as well but for now, we will try doing it for flat-type.
 +
|-
 +
| <p><center> '''OneMap Visualization of resale flat prices''' </center></p>
 +
[[File:Onemap screenshot.png|400px|center]]
 +
<p><center> '''Source''': https://www.onemap.sg/main/v2/propertyquery </center></p>
 +
||
 +
* This is a screenshot of an implementation of a interactive map by OneMap which allows users to search for resale flats on the map and view information related to the selected flat easily.
 +
* We found this very good as it was very informative if you only wanted to look at one flat.
 +
* However, the map looks very ugly and out-dated in terms of style and coloring and this map does not allow one to easily do comparisons between flats.
 +
|}
 +
 +
 +
<!-- START PROTOTYPE -->
 +
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px">Prototype </div>==
 +
 +
<!-- END  PROTOTYPE  -->
  
 
<!-- START Project Schedules -->
 
<!-- START Project Schedules -->
Line 42: Line 174:
 
<!-- END  Project Schedules  -->
 
<!-- END  Project Schedules  -->
  
 +
<!-- START CHALLENGES -->
  
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px">Challenges</div>==
  
<!-- References -->
+
<!-- END  CHALLENGES  -->
 +
{| class="wikitable" style="background-color:#FFFFFF;" width="100%"
 +
|-
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;width: 30%;" | Challenges
 +
! style="font-weight: bold;background: #000000;color:#ffcc33;" | Possible Solutions
 +
|-
 +
|
 +
<p>Unfamiliar with D3.js </p>
 +
||
 +
* Independent learning through online learning resources
 +
* Validating learning outcome through review and coding practices
 +
|-
 +
| <p>Data Merge, Cleaning and Transformation</p>
 +
||
 +
* Subzone energy usage data:
 +
* Missing NA records: government have purposedly removed some data points to enforce the data privacy. We will be examine the effect of remove the NA and decide the appropriate action to take.
 +
|-
 +
| <p>Choice of web hosting provider</p>
 +
||
 +
* A quick production pipeline required due to the time limit
 +
* Examine the requirement of the data visualisation: dynamic or statics
 +
* Current solution is to use Github Page as a hosting provider there is no dynamic data retrieval required
 +
|-
 +
| <p>Unfamilar with implementation efforts required for customized D3.js interactivity</p>
 +
||
 +
* The week will be spending 2 weeks to familiarize with D3.js structure & syntax
 +
* Follwing 2 weeks will be trying out the customized D3.js interactivity
 +
* The project scope and plan will be re-examined based on the project objective, complexity and time available
  
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px"> References  </div>==
+
|}
  
<!-- END  References   -->
+
<!-- START References -->
 
 
===Technology Used===
 
  
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px"> References</div>==
  
 +
<!-- END  References    -->
  
 +
[1] Energy Market Authority (https://www.ema.gov.sg/singapore_energy_statistics.aspx) <br />
 +
[2] Data Gov Database (https://data.gov.sg) <br />
 +
[3] D3.js (Documentation https://d3js.org/) <br />
 +
[4] Observalehq (https://beta.observablehq.com/) <br />
 +
[5] One Map (https://www.onemap.sg/main/v2/) <br />
 +
[6] Energy Consumption Predition Example (http://cs109-energy.github.io/building-energy-consumption-prediction.html) <br />
  
===References===
+
<!-- START Feedback -->
[1]
 
  
 +
==<div style="margin-top: 6px;font-weight:bold;text-align: left;font-size:20px; border-bottom:5px solid #ffcc33;text-align:center; background-color: #000000; color: #ffcc33; padding: 2px"> Feedback</div>==
 +
Please feel free leave your comments, suggestions or anything interesting :)
  
Other Reference List:
+
<!-- END  Feedback    -->

Latest revision as of 19:04, 14 October 2018

IS428_ChargeMetrics_Project
HOME  

PROPOSAL

  PROJECT POSTER  

PROJECT APPLICATION

  RESEARCH PAPER


Motivation

Objectives

Data

Dataset/Source Data Attributes Rationale Of Usage
Resales flat prices from Mar 2012 onwards
(https://data.gov.sg/dataset/resale-flat-prices?resource_id=83b2fc37-ce8c-4df4-968b-370fd818138b )
Resales flat prices from 2002 - Feb 2012
(https://data.gov.sg/dataset/resale-flat-prices?resource_id=8c00bf08-9124-479e-aeca-7cc411d884c4 )
Resales flat prices from 1990 - 1999
(https://data.gov.sg/dataset/resale-flat-prices?resource_id=adbbddd3-30e2-445f-a123-29bee150a6fe )
  • Data Type 1
  • Data Type 1
This dataset will be used as a main source of information in our analysis to understand the number of HDB around Singapore from 1990 to 1999, 2002 to Feb 2012 and Mar 2012 onwards respectively.
Bus Stop Names and Locations
(https://www.mytransport.sg/content/mytransport/home/dataMall.html#)
  • Bus Stop Number
  • Bus Stop Roof Number
  • Bus Stop Name
  • X
  • Y
  • Latitude
  • Longitude
This dataset aims to complement the main dataset by providing detailed information about the latitude and longitude of the bus stops located around HDB. We use a javascript geocoding script to convert all the X and Y coordinates to EPSG:4326 latitude and longitude coordinates.
Mrt Stations Names and Locations
(https://www.mytransport.sg/content/mytransport/home/dataMall.html#)
  • MRT Station Number
  • MRT Station Name
  • X
  • Y
  • Latitude
  • Longitude
This dataset aims to complement the main dataset by providing detailed information about the latitude and longitude of the MRT stations located around HDB. We use a javascript geocoding script to convert all the X and Y coordinates to EPSG:4326 latitude and longitude coordinates
Master Plan Subzone Boundary Names and GeoPolygon
(https://data.gov.sg/dataset/master-plan-2014-subzone-boundary-no-sea)

  • Polygon
  • Name
  • Subzone Number
  • Subzone Code
  • Region Name
  • Area Code
  • Area Indicator
This dataset aims to complement the main dataset by providing detailed information about all the subzone in Singapore. We use a javascript library toGeoJson.js to help us convert .KML file to .GeoJson file


Related Works

The following related work

There are many charts and visualisations available which illustrates the various trends of house prices and index. We have selected a few of these to study and learn before we begin developing our own visualizations.

Related Works What We Can Learn

An Analysis of the trend and correlation between resale prices and flat production

Source: http://www.teoalida.com/singapore/hdbprices/

  • The use of 2 different chart types with a secondary axis is effective in illustrating the correlation between resale prices and flat production.
  • The colours used are striking and contrast well with each other.
  • There are dips in both variables which are not explained in the infographic itself (E.g. 1997 Asian crisis, 2003 SARs outbreak). This events could be incorporated into the charts to make it more informative.
  • However, major events are not shown in the chart itself. This makes it difficult for the audience to understand the various changes throughout the years. It would be much clearer if events were annotated directly on the charts.

An interactive heatmap of Singapore’s house prices in various districts

Source: https://www.srx.com.sg/heat-map

  • This heatmap uses colours appropriately so that the house prices of each district can be identified intuitively (Red means expensive, blue means cheap, orange means mid-range)
  • The use of filters allows user to find out more about the price distribution of each house type easily.
  • When user mouseover a district on the heatmap, the corresponding district on the legend is highlighted. This improves usability as users do not have to match district numbers manually.
  • Even though the colours are intuitive, it could still be confusing as audience might still have to constantly refer back to the colour scale to interpret the colours. Colour intensity of a single colour could be used instead of to illustrate the difference in prices.

An interactive visualization of house prices along MRT stations

Source: https://www.srx.com.sg/mrt-home-prices/property-listings-near-east-west-line

  • This visualization makes use of unique ways to illustrate the relation between nearby facilities and house prices. Thus in a glance, a person can tell where are the expensive areas based on the MRT stations.
  • The separating of the various MRT lines using filters at the top prevent too much information from being shown in one page
  • However, this chart could be too high level and general. Illustrating this on the Singapore map would be more useful instead, where each house's proximity to the MRT station and their respective resale price is shown. Also, in order to drill down, you would have to navigate to another page and then search for a specific house based on a table view which does not help one visualize where the resale flat is actually located at.

Four Ways to Slice Obama’s 2013 Budget Proposal

Source: http://www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html

  • This visualization makes use of unique ways to illustrate Obama's Budget Proposal
  • The various circles denote the various types of proposed budget items and the circles are sized based on the proposed budget amount.
  • Users can toggle the links to slice and dice the initial visualization into different views so that a user can get different insights from the same initial visualization.
  • This chart is very interesting and it is something that we wanted to use to present the changes in resale prices based on different flat types. If we have the time, we could do different filters as well but for now, we will try doing it for flat-type.

OneMap Visualization of resale flat prices

Source: https://www.onemap.sg/main/v2/propertyquery

  • This is a screenshot of an implementation of a interactive map by OneMap which allows users to search for resale flats on the map and view information related to the selected flat easily.
  • We found this very good as it was very informative if you only wanted to look at one flat.
  • However, the map looks very ugly and out-dated in terms of style and coloring and this map does not allow one to easily do comparisons between flats.


Prototype

Project Schedules

Challenges

Challenges Possible Solutions

Unfamiliar with D3.js

  • Independent learning through online learning resources
  • Validating learning outcome through review and coding practices

Data Merge, Cleaning and Transformation

  • Subzone energy usage data:
  • Missing NA records: government have purposedly removed some data points to enforce the data privacy. We will be examine the effect of remove the NA and decide the appropriate action to take.

Choice of web hosting provider

  • A quick production pipeline required due to the time limit
  • Examine the requirement of the data visualisation: dynamic or statics
  • Current solution is to use Github Page as a hosting provider there is no dynamic data retrieval required

Unfamilar with implementation efforts required for customized D3.js interactivity

  • The week will be spending 2 weeks to familiarize with D3.js structure & syntax
  • Follwing 2 weeks will be trying out the customized D3.js interactivity
  • The project scope and plan will be re-examined based on the project objective, complexity and time available


References

[1] Energy Market Authority (https://www.ema.gov.sg/singapore_energy_statistics.aspx)
[2] Data Gov Database (https://data.gov.sg)
[3] D3.js (Documentation https://d3js.org/)
[4] Observalehq (https://beta.observablehq.com/)
[5] One Map (https://www.onemap.sg/main/v2/)
[6] Energy Consumption Predition Example (http://cs109-energy.github.io/building-energy-consumption-prediction.html)


Feedback

Please feel free leave your comments, suggestions or anything interesting :)