Difference between revisions of "Group02 proposal"

From Visual Analytics for Business Intelligence
Jump to navigation Jump to search
 
(110 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
[[File:Rain & Shine(new).png|center|300px]]
 +
 
<!--Header-->
 
<!--Header-->
 
<div style="width:100%; text-align:center;">
 
<div style="width:100%; text-align:center;">
 
{|style="background-color:#143c67; color:#4d79ff; padding: 10 0 10 0;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0"  |
 
{|style="background-color:#143c67; color:#4d79ff; padding: 10 0 10 0;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0"  |
 
| style="padding:0.2em; font-size:100%; background-color:#143c67; text-align:center; color:#F5F5F5" width="10%" |  
 
| style="padding:0.2em; font-size:100%; background-color:#143c67; text-align:center; color:#F5F5F5" width="10%" |  
<font color="#F5F5F5" size=3 face="Helvetica">Wolf of HDB Street</font>
+
;
 +
[[Group02 team|<font color="#F5F5F5" size=3 face="Helvetica">Team</font>]]
  
 
| style="background:none;" width="1%" | &nbsp;
 
| style="background:none;" width="1%" | &nbsp;
| style="padding:0.2em; font-size:100%; background-color:#05050f;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
+
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
;
 
;
[[Group08 proposal| <font color="#FFFFFF">Proposal</font>]]
+
[[Group02 proposal v2| <font color="#FFFFFF">Proposal</font>]]
  
 
| style="background:none;" width="1%" | &nbsp;
 
| style="background:none;" width="1%" | &nbsp;
 
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
;
 
;
[[Group08 poster| <font color="#FFFFFF">Poster</font>]]
+
[[Group02 poster| <font color="#FFFFFF">Poster</font>]]
  
 
| style="background:none;" width="1%" | &nbsp;
 
| style="background:none;" width="1%" | &nbsp;
 
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
;
 
;
[[Group08 application| <font color="#FFFFFF">Application</font>]]
+
[[Group02 application| <font color="#FFFFFF">Application</font>]]
  
 
| style="background:none;" width="1%" | &nbsp;
 
| style="background:none;" width="1%" | &nbsp;
 
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
| style="padding:0.2em; font-size:100%; background-color:#143c67;  border-bottom:0px solid #3D9DD7; text-align:center; color:#F5F5F5" width="10%" |  
 
;
 
;
[[Group08 research paper| <font color="#FFFFFF">Research Paper</font>]]
+
[[Group02 research paper| <font color="#FFFFFF">Research Paper</font>]]
 
|}
 
|}
  
 
</div>
 
</div>
 
<!--/Header-->
 
<!--/Header-->
 
+
<div>
 
+
{|style="background-color:#143c67; color:#143c67; padding: 10 0 10 0;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0"  |
<div></div>
+
| style="padding:0.2em; font-size:100%; background-color:#ff6b89; text-align:center; color:#F5F5F5" width="10%" |
 +
;
 +
[[Group02 proposal v2|Version2]]|[[Group02 proposal|Version1]]
 +
|}
 +
</div>
 
<br /><br />
 
<br /><br />
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>PROBLEM & MOTIVATION</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>PROBLEM & MOTIVATION</center></font></div>==
  
 
'''<u>Problem</u>''' <br>
 
'''<u>Problem</u>''' <br>
As a buyer looking for Resale HDB flats, it can be difficult to make a purchase decision due to the lack of information in the market. Information such as increasing or decreasing price trends over the years for each estate (e.g. Tampines) or submarket (e.g. 4-ROOM flats) could be essential in the decision making process.  
+
When it comes to purchasing or renting a property, there are many factors that go into a buyer’s consideration before he makes the final decision. The primary concern for buyers is the pricing of the property [https://www.ura.gov.sg/Corporate/Property/Residential/Buying-Property]. However, our group identified that there are also secondary concerns such as the weather and the amenities available that do influence the buyer’s final decision to purchase the property. There are limited tools available to help property buyers to identify areas that suit their needs/preferences best. The current available tools are only optimal to suit one category of concern, but fails when we try to use more categories to make our visualization. <br><br>
 +
For example, different people have different preferences for the weather. Some may prefer sunny weather while others prefer it to rain all the time. Currently, there are tools to find weather information, and property prices information, but not both at the same time.<br><br>
  
 
'''<u>Motivation</u>''' <br>
 
'''<u>Motivation</u>''' <br>
According to Ms. Christine Sun, head of research and consultancy at OrangeTee, She commented in November last year (2019) that [https://www.straitstimes.com/singapore/more-hdb-resale-flats-sold-in-october-after-higher-housing-grants-income-ceilings-kicked demand for HDB resale flats has been strengthening] in the recent months. However, our group felt that the statement was too generalised as there are several submarkets in the resale of HDB flats such as 3-ROOM flats and 5-ROOM flats just to name a few. [https://www.businesstimes.com.sg/hub-projects/property-2019-september-issue/hdb-resale-market-sees-strong-demand Each submarket could have a different trend]. Additionally, trends could also vary across different estates such as Bukit Merah and Tampines. The information online would not be useful for people looking at specific submarkets in certain estates.
+
Through our visualization, the group hopes to assist users to be able to easily visualize a home of his dreams in different aspects beyond mere prices. Our visualization will incorporate data from property prices, weather and amenities to identify the accessibility of the area and help our users to justify the real value of the property. With our visualization dashboard, users will be able to identify properties that they truly desire.
  
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>OBJECTIVES</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>OBJECTIVES</center></font></div>==
<b>Target Group: </b> Resale flat buyers <br>
+
 
Our goal in this project is to design and create an interactive one-stop visualization tool that could provide Resale flat buyers with information such as: <br>
+
We aim to provide an interactive visualization dashboard to assist <b>pGeneral Public, people living in Singapore</b> with understanding the weather of our country with visualization information such as:
<ul>
+
 
<li> Changes of flat prices over time for each submarket by estate (e.g. 4-ROOM flats price changes over the past 5 years for Ang Mo Kio) </li>
+
:# Insights on the Rainfall Precipitation of the whole of Singapore and each postal area for the past 20 years.
<li> High and low value estates based on past prices (e.g. Tampines is a low value estate based on prices from the past 5 years) </li>
+
:# Insights on the Temperature of the whole of Singapore and each postal area for the past 20 years.
<li> Changes in resale prices based on remaining lease (i.e. age of the estate) for each estate </li>
+
:# Insights on the relationship of the Rainfall Precipitation and Temperature in the different months/seasons yearly.
<li> Distribution of flat prices for each submarket and estate </li>
+
 
</ul>
+
<b><u>Target Group</u></b>:
These information would help buyers make better purchase decision(s).
+
* <b> General Public, people living in Singapore
  
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>DATASET</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>DATASET</center></font></div>==
 
<center>
 
<center>
 +
The Data Sets we will be using for our analysis and for our application is listed below:
 
{| class="wikitable" width="100%"
 
{| class="wikitable" width="100%"
 
|-
 
|-
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 30%;" |Data/Source
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 30%;" |Data/Source
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 30%;" |Variables/Description
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 30%;" |Variables/Description
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 40%;" |Methodology
+
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 40%;" |Rationale Of Usage
 
|-
 
|-
 
|
 
|
 
<center>
 
<center>
Resale Flat Prices (January 1, 2017 to January 31, 2020) <br>
+
<b>Temperature and Rainfall Data</b><br/>
Taken from: Data.gov.sg <br>
+
(Jan 2012 - Dec 2019)<br/><br/>
[https://data.gov.sg/dataset/resale-flat-prices Link to Data Source]
+
 
 +
(http://www.weather.gov.sg/climate-historical-daily/)
 
</center>
 
</center>
 +
||
 +
* Stations
 +
* Date
 +
* Daily Rainfall
 +
* Highest  30-min/60-min/120-min Rainfall (mm)
 +
* Mean/Minimum/Maximum Temperature (°C)
 +
* Mean/Max Wind Speed (km/h)
 +
||
 +
<center>
 +
This dataset covers a good time series of Singapore's weather from 2012 to 2019 across different weather categories. Our team wish to spot the trend or pattern of Singapore's climate in every town if possible.
 +
</center>
 +
|-
 
|
 
|
<ol>
+
<center>
<li> <b> month: </b> Transacted Year & Month  </li>
+
<b>Amenities Location Data</b></br><br/>
<li> <b> Town: </b> Town the flat is situated in </li>
+
(https://api.data.gov.sg/v1/environment/rainfall)
<li> <b> Flat Type: </b> Type of Housing </li>
+
(https://api.data.gov.sg/v1/environment/air-temperature)
<li> <b> Block: </b> Identifier for each Housing </li>
+
</center>
<li> <b> Street Name: </b> Identifier for each Street </li>
+
||
<li> <b> Storey Range: </b> Range of Storey the flat is situated in </li>
+
* Station ID
<li> <b> Floor Area Sqm: </b> Size of flat </li>
+
* Station Name
<li> <b> Flat Model: </b> Flat Model </li>
+
* Latitude
<li> <b> Lease Commence Date: </b> Start Date of Lease </li>
+
* Longitude
<li> <b> Remaining Lease: </b> Duration remaining for Lease </li>
+
||
<li> <b> Resale Price: </b> Price the Flat is sold for </li>
+
<center>
</ol>
+
The data set will be used to anchor the amenities available for the selected property in a specified range
|
+
</center>
Obtain information on flat prices by:
+
 
<ul>
 
<li> Town </li>
 
<li> Flat Type </li>
 
<li> Town & Flat Type (e.g. 4-ROOM in Tampines vs. 4-ROOM in Bedok) </li>
 
<li> Town & Block (e.g. 105 TAMPINES vs. 115 TAMPINES) </li>
 
<li> Storey Range </li>
 
<li> Floor Area </li>
 
<li> Town & Floor Area </li>
 
<li> Remaining Lease </li>
 
</ul>
 
The list is non exhaustive, more could be added in the future.
 
 
|-
 
|-
 
|}
 
|}
Line 98: Line 109:
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>BACKGROUND SURVEY OF RELATED WORK</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>BACKGROUND SURVEY OF RELATED WORK</center></font></div>==
  
In order for our group to design a new visualisation, it was important to us that we understand the current work out there in the field. This will enable us to make informed decisions on developing our own visualisations. We can also learn from the current visualisations to ensure that our own work adds value and to not repeat any mistakes made. Listed below are screenshots of visualisations and their learning points respectively.
+
Below are a few visualizations and charts we considered making for our projects.
 
 
 
 
 
{| class="wikitable" style="margin-left: auto; margin-right: auto; width: 90%;
 
{| class="wikitable" style="margin-left: auto; margin-right: auto; width: 90%;
 
|-
 
|-
! style="background: #899db3;color:#fbfcfd;|Reference of Other Interactive Visualization !!style="background: #899db3;color:#fbfcfd;| Learning Points
+
! style="background: #899db3;color:#fbfcfd;|Visual Considerations !!style="background: #899db3;color:#fbfcfd;| Insights / Comments
 
|-
 
|-
| <center><br/>  '''Title: Official HDB Map Services '''  
+
| <center><br/>  '''Title: Qualitative Thematic Map '''
[[File:Image5.jpg|thumb|Image5| 500px |center]]
+
[[File:ThematicMap.png|300px|frameless|center]]
Source: https://services2.hdb.gov.sg/web/fi10/emap.html
+
<b>Source: https://mapdesign.icaci.org/2014/12/mapcarte-353365-life-in-los-angeles-by-eugene-turner-1977/</b>
 
</center>
 
</center>
 
||  
 
||  
* This is an interactive visualisation by HDB that we can search and filter different regions
+
 
* This visualisation is quite messy as icons of all current HDB in a particular is shown, and the user might be confused to which house to pick from. Furthermore we are not able to understand the price changes across time
+
One of the items that we looked at is this qualitative thematic map that was covered in class.
* The advantage of this visualisation is being able to visualise the clustering of HDB flats in a particular region
+
 
 +
From our initial brainstorming of ideas, we intend to look at various factors that a buyer will look at, giving the buyer a high-level overview of the different areas and whether it fits the criteria that he chooses. How we will adapt ideas from this graph is for us to allow the users to make a few selections of multiple factors. Then based on which criteria the different properties in the different subzones can meet, we can choose different shapes, colours to represent the zone.
 +
 
  
 
|-
 
|-
| <center><br/> '''Title: Average HDB resale prices by town treemap '''
+
| <center><br/> '''Title: Whisker plot of temperature'''
[[File:Image3.jpg|thumb|Image3| 500px |center]]
+
[[File:Temperature whisker.png|300px|frameless|center]]
Source: http://sgyounginvestment.blogspot.com/2018/03/visualisation-of-hdb-resale-prices-in.html
+
<b>Source: https://www.ck12.org/statistics/box-and-whisker-plots/rwa/The-Ways-of-Weather/</b>
 
</center>
 
</center>
 
||  
 
||  
* This is a heatmap that shows the relationship of average resale prices by towns
+
We can see the temperature for the selected area over a year. The whisker plots can show the upper and lower boundaries of temperature, and we can observe that the temperature gradually rises to a peak from Jan to Aug, before decreasing until December.
* One further improvement that we can do to this visualisation is to add in subcategories of the different resale prices. These subcategories could be type of HDB resale flats and which storeys they are on
+
 
 +
We hope to apply this chart to display the rainfall for a selected area over a year. This allows buyers to be able to better understand the rainfall pattern in the area so that he can better understand if the area suits his preferences.  
  
 
|-
 
|-
| <center><br/>  '''Title: Distribution of Past HDB Transactions'''  
+
| <center><br/>  '''Title: Heatmap of rainfall'''  
[[File:Image7.png|thumb| 500px |center]]
+
[[File:Heatmap of rainfall.png|300px|frameless|center]]
Source: https://hdbviz.shinyapps.io/hdbviz/
+
<b>Source: https://www.shanelynn.ie/analysis-of-weather-data-using-pandas-python-and-seaborn/</b>
 
</center>
 
</center>
 
||  
 
||  
* This is a highlight table that can be used to depict the distribution of price compared to region and flat type. Furthermore there is a more detailed box plot at the side that visualises the range of price
+
This is a heatmap of daily rainfall. Darker colours of red represent heavier rainfall.  
* One improvement that can be made to this visualisation is labelling the highlight table to include the prices of each cell, this is to give clarity by showing the magnitude of the price
 
* One other improvement that can be made to this visualisation is to allow the user to have an option to include volume of sales as well.
 
  
 +
Another way to have a visualization to understand the patterns of rainfall. Through this, we can quickly see how many days in a year where there is rain for a selected subzone. Assuming that a potential buyer is interested in a property that sees more sunlight, he will be more interested in a subzone where the graph looks brighter. On the other hand, if the buyer is interested in a property that is always rainy, he would be interested in an area where the graph looks darker.
  
 
|-
 
|-
| <center><br/>  '''Title: Distribution of 4-Room HDB Resale Prices By Town '''  
+
| <center><br/>  '''Title: Spatial Interpolation'''  
 
+
[[File:Property heatmap.png|300px|frameless|center]]
[[File:Image2.jpg|thumb|500px|center]]
+
<b>Source: https://www.srx.com.sg/heat-map/</b>
Source: https://medium.com/@wojiefu/hdb-pusle-visualization-of-singapore-hdb-flat-resale-records-2e2fbedbee91
 
 
</center>
 
</center>
 
||  
 
||  
* This is a animated visualisation of HDB resale prices on the map of Singapore, it is very effective in showing us the changes in number of occurrence of transactions being made at what frequency
+
This graph shows the property prices in Singapore for different property types. The user can choose to select different property types, and the graph will update to show only the selected property type. A variety of colours are chosen here to display different levels of prices.  
* A disadvantage of this visualisation is that there is no legend or information to relate the colours of the points and the actual resale price
 
* A disadvantage of this visualisation is that there is no clear indication of which region belongs to which section in the geography map of Singapore. This leaves the user with the onus to understand the location in Singapore
 
  
 +
Based on our problem, there are 2 key aspects that we are looking at: Prices and Weather. One way this kind of visualization could be utilized by our group is for us to use this to display prices or Weather in Singapore across all subzones. By charting either prices or the various weather types over a map of Singapore, the user will be able to quickly gain an understanding of how the different criteria that he can choose will be like across Singapore.
  
|}
 
  
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>REFERENCE LIST</center></font></div>==
 
'''<u>References</u>''' <br>
 
<ol>
 
<li> https://www.straitstimes.com/singapore/more-hdb-resale-flats-sold-in-october-after-higher-housing-grants-income-ceilings-kicked </li>
 
<li> https://www.businesstimes.com.sg/hub-projects/property-2019-september-issue/hdb-resale-market-sees-strong-demand </li>
 
<li> https://www.reddit.com/r/singapore/comments/dubsyk/visualising_30_years_of_hdb_resale_flat_prices/ </li>
 
<li> https://medium.com/@wojiefu/hdb-pusle-visualization-of-singapore-hdb-flat-resale-records-2e2fbedbee91 </li>
 
<li> http://sgyounginvestment.blogspot.com/2018/03/visualisation-of-hdb-resale-prices-in.html </li>
 
<li> https://services2.hdb.gov.sg/web/fi10/emap.html </li>
 
<li> https://hdbviz.shinyapps.io/hdbviz/ </li>
 
  
  
</ol>
+
|}
  
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>KEY TECHNICAL CHALLENGES & MITIGATION</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>KEY TECHNICAL CHALLENGES & MITIGATION</center></font></div>==
Line 169: Line 167:
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 15%;" |Challenge
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 15%;" |Challenge
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 40%;" |Description
 
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 40%;" |Description
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 40%;" |Mitigation
+
! style="font-weight: bold;background: #899db3;color:#fbfcfd;width: 40%;" |Mitigation Plan
 +
|-
 +
|<center>1. </center>
 +
||Software Challenge
 +
||Unfamiliarity of visualisation tools such as R, R Shiny, Tableau.
 +
||
 +
* Github Learning
 +
* Stackoverflow research
 +
* Self-directed and peer learning
 +
* Watch video tutorials from YouTube
 +
* Hands-on practice using the different training platforms such as Data Camps
 +
|-
 +
|<center>2. </center>
 +
||Programming Challenge
 +
||Inexperince with data cleaning and transformation using R
 +
||
 +
* Trial and error
 +
* Read online articles and forums for guidance
 +
* Watch video tutorials on how to fully utilise packages such as lapply, tidyr and dplyr
 
|-
 
|-
| 1.
+
|<center>3. </center>
| Lack of Familiarity with Tools
+
||Workload Constraint
| Everyone in the group do not know how to program in RShiny for visualisation
+
||Time and Workload Constrains
| We will learn Rshiny during class, call for consultation and rely on Googling for any programming challenges. Alternatively, there is also Datacamp available for us.
+
||
 +
* Design reasonable project timeline based on everyone's ability and capacity.
 +
* Set milestones and adjust the timeline accordingly based on the team's progress.
 
|-
 
|-
| 2.
+
|<center>4. </center>
| Viability of Ideas
+
|| Dataset Complexity
| We do not know if the current dataset is sufficient in providing all the information needed to conduct analysis and building of planned visualizations.
+
||
| There are multiple dataset online to use and we can use Prof Kam's REALIS dataset provided to us to supplement our dataset if we are lacking of certain variables. We could also derive our own variables based on the current dataset if needed (e.g. Geocoding).
+
Our have different data from multiple sources in multiple different formats, hence we foresee a huge challenge in standardizing the data
 +
* Note: Our current dataset is looking at 49 areas over the spread of 8 years of data, for every year there are 12 months of data. This gives a total of 4,704 CSV files to consolidate and clean for weather data alone.
 +
||
 +
* Make use of data preparation tools such as tableau prep
 +
* Make use of our database management skills to normalize all data tables into third normal form
 
|-
 
|-
| 3.
 
| Lack of Domain Knowledge
 
| HDB resale prices are affected by a spectrum of different factors such as policy measures and redevelopment. It is hard for us to understand without domain knowledge.
 
| Learn from informative websites such as from HDB and iteratively discover and learn insights into the dataset
 
 
|}
 
|}
</center>
 
  
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>STORYBOARD</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>STORYBOARD</center></font></div>==
Line 194: Line 211:
 
! style="background: #899db3;color:#fbfcfd;| Dashboards !!style="background: #899db3;color:#fbfcfd;| Description
 
! style="background: #899db3;color:#fbfcfd;| Dashboards !!style="background: #899db3;color:#fbfcfd;| Description
 
|-
 
|-
| <center><br/>  ''' Dashboard 1: Overall Price Distribution by Submarket and Submarket Sizes '''  
+
| <center><br/>  ''' Dashboard 1: Qualitative Thematic Map of Singapore property'''  
[[File:Dashboard1 1.png|thumb|500px|center]]
+
[[File:VA1.jpg|400px|frameless|center]]
[[File:Dashboard1 2.png|thumb|500px|center]]
 
 
</center>
 
</center>
 
||  
 
||  
* The first dashboard that we would like to envision is to have a filter that is visualised by the Singapore map. This will allow users to intuitively find an area that they are familiar with. Since this is only just a filter, it will not show the distribution of HDB Resale Flats in the map itself.
 
  
* By clicking on a specific town, the dashboard will dynamically change (a) and provide insights of its submarket price distribution. This will be visualised in the form of a box plot.
+
Our group plans to do an interactive Qualitative Thematic Map which reflects different faces based on the year, price, weather, amenities filters adjusted by the users. This chart will show the data at a high level for users to identify which area meets their needs in a glance. Users can also view the map based on the postal area or zone.  
  
* Furthermore, by clicking on a specific box plot, it is used as a filter for graph (b), where there will be a histogram plot for the volume of different psf sizes of the submarkets.
+
Filters used includes:
 +
* Sliders
 +
:# Year
 +
:# Transacted Price
 +
:# Amenities
 +
* Single Dropdown List
 +
:# Weather
 +
:# Map level of detail
 +
* Multiple Drowndown List
 +
:# Property Type
 +
 
 +
 
 +
Based on User’s adjustment for the filters, the map would reflect the user <u>3 different data types</u>:
 +
* <b>Weather</b>
 +
: When the postal area/zone’s selected weather is above the median of Singapore’s data over the selected year, the chart will reflect a circle face shape. However, if the postal area/zone’s selected weather falls in the median or is lower than the median of Singapore’s data over the selected year, the chart will reflect a circle face shape with devil horns.
 +
* <b>Pricing</b>
 +
: When the postal area/zones have houses with pricing over the selected year that meets the requirements of the user based on his/her filter, the chart will reflect a smiling face. However, if the postal area/zones have houses with pricing over the selected year that is lower than the requirements of the user based on his/her filter, the chart will reflect a blank face. Lastly, if the postal area/zones have houses with pricing over the selected year that is more than the requirements of the user based on his/her filter, the chart will reflect a sad face.
 +
* <b>Amenities</b>
 +
: When the postal area/zone’s number of amenities in the selected year meets the requirements of the user based on the filter, the chart will reflect the faces to be shown in green colour. However, if the postal area/zone’s number of amenities does not meet the requirements of the user, the chart will reflect the faces in blue colour.
 +
 
 +
 
 +
With this visualisation, users will be able to identify and shortlist the area(s) that meets their requirements the best.  
  
 
|-
 
|-
| <center><br/>  ''' Dashboard 2 '''  
+
| <center><br/>  ''' Dashboard 2: Property prices in Postal Areas{Based on Users selected areas in storyboard 1} '''  
[[File:Dashboard2.png|thumb|500px|center]]
 
 
</center>
 
</center>
 +
[[File:VA2.jpg|400px|frameless|center]]
 
||  
 
||  
* Description 1
+
 
* Description 2
+
The purpose of this chart is to show a detailed breakdown of the properties that meet the requirements of the Users based on his/her filters and User's shortlisted area(s) in the chart from Dashboard 1.
* Description 3
+
 
 +
 
 +
This chart shows all the properties and their prices for all the properties that meet the requirements of the Users based on the filter’s range and the shortlisted area(s).
 +
 
 +
 
 +
Filters used includes:
 +
*Sliders
 +
:# Year
 +
:# Transaction Price
 +
:# Amenities
 +
* Single Dropdown List
 +
:# Property Type
 +
:# Sorted By
 +
 
 +
 
 +
The chart will be able to help users better understand the property prices based on his/her shortlisted area(s) and make a decision on which area’s property to purchase.
 +
 
 +
 
 +
: <b>X-Axis:</b> Transacted Properties’ Name<br/>
 +
: <b>Y-Axis:</b> Transacted Pricing
 +
 
 +
 
 +
This chart will be shown together with the chart on Dashboard 3 to help the buyer make the best-informed decisions.
  
 
|-
 
|-
| <center><br/>  ''' Dashboard 3: Tree Map of HDB Storeys by Volume and Price '''  
+
| <center><br/>  ''' Dashboard 3: Distribution of Rain Precipitation Amount/ Temperature/ Wind Speed in Postal Areas{Based on Users selected areas in Dashboard 1} '''  
[[File:Dashboard3.png|thumb|500px|center]]
 
 
</center>
 
</center>
 +
[[File:VA3.jpg|400px|frameless|center]]
 +
||
 +
 +
The purpose of this chart is to show a detailed breakdown of the weather (which includes Rain Precipitation Amount/ Temperature/ Wind Speed) for User’s shortlisted area(s) from the chart in Dashboard 1.
 +
 +
 +
Filters used includes:
 +
* Sliders
 +
:# Year
 +
:# Month
 +
* Single Dropdown List
 +
:# Level of Detail
 +
 +
 +
The user can adjust the filters to identify any patterns or trends of the weather based on the short-listed areas in Singapore. This chart can help the user better identify which area best suits the user based on his preferences and needs.
 +
 +
 +
: <b>X-Axis:</b> Level of Detail(Sub-zone/ Postal Area/ Zone)<br/>
 +
: <b>Y-Axis:</b> Rain Precipitation Amount/ Temperature/ Wind Speed
 +
 +
 +
This chart will be shown together with the chart on Dashboard 2 to help the buyer make the best-informed decisions.
 +
 +
|-
 +
| <center><br/>  ''' Dashboard 4: Comparing Rainfall {selected weather} and the median pricing of All Properties{selected property type} '''
 +
</center>
 +
[[File:VA4.jpg|400px|frameless|center]]
 
||  
 
||  
* The final dashboard that we envision to complete is a Tree Map plot to visualise the volume and price of HDB Resale flats by their Storeys. The size of the boxes will be equivalent to the volume transacted value and the colours will represent their respective price.
+
 
* The dashboard will come with filters that enables users to select the particular towns that they will want to compare with. The maximum allowed number to select is up to 2 towns. The number of towns selected will generate the same number of graphs for visualisation.
+
The chart in this storyboard reflects a combination of two thematic maps, a bar chart map as well as a spatial interpolation map. Similar to the chart in Dashboard 1, this chart will show data at a high level for users to identify which area meets their needs in a glance. However, this chart is more-straightforward as the results shown in this chart is more clear cut and less informative.
* Lastly, one other filter is the submarkets in which the user is able to select the type of submarket for deeper analysis.  
+
 
 +
 
 +
In this chart, the user will be able to compare:
 +
* The weather of the user’s choice in each zone/postal area.
 +
* Median pricing of their selected property type in each zone/postal area.
 +
 
 +
 
 +
Filters used includes:
 +
* Sliders
 +
:# Year
 +
* Single Dropdown List
 +
:# Level of Detail
 +
:# Weather
 +
:# Property Type
 +
 
 +
 
 +
With the visualization, users will be able to compare all the areas in Singapore to make an informed decision. By doing so, the chart can help the user in shortlisting the area that they wish to zoom into in another view of light as compared to the chart in Dashboard 1 where detailed comparison of the whole Singapore is limited.
 +
 
 
|}
 
|}
  
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>MILESTONES</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>MILESTONES</center></font></div>==
[[Image:Milestones.jpg | frameless | center | 500px ]]
+
[[File:Photo 2020-03-01 16-22-33.jpg|1000px|frameless|center]]
  
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>COMMENTS</center></font></div>==
 
==<div style="background:#143c67; padding: 15px; font-weight: bold; line-height: 0.3em; letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Century Gothic"><center>COMMENTS</center></font></div>==

Latest revision as of 15:32, 11 April 2020

Rain & Shine(new).png

Team

 

Proposal

 

Poster

 

Application

 

Research Paper

Version2|Version1



PROBLEM & MOTIVATION

Problem
When it comes to purchasing or renting a property, there are many factors that go into a buyer’s consideration before he makes the final decision. The primary concern for buyers is the pricing of the property [1]. However, our group identified that there are also secondary concerns such as the weather and the amenities available that do influence the buyer’s final decision to purchase the property. There are limited tools available to help property buyers to identify areas that suit their needs/preferences best. The current available tools are only optimal to suit one category of concern, but fails when we try to use more categories to make our visualization.

For example, different people have different preferences for the weather. Some may prefer sunny weather while others prefer it to rain all the time. Currently, there are tools to find weather information, and property prices information, but not both at the same time.

Motivation
Through our visualization, the group hopes to assist users to be able to easily visualize a home of his dreams in different aspects beyond mere prices. Our visualization will incorporate data from property prices, weather and amenities to identify the accessibility of the area and help our users to justify the real value of the property. With our visualization dashboard, users will be able to identify properties that they truly desire.

OBJECTIVES

We aim to provide an interactive visualization dashboard to assist pGeneral Public, people living in Singapore with understanding the weather of our country with visualization information such as:

  1. Insights on the Rainfall Precipitation of the whole of Singapore and each postal area for the past 20 years.
  2. Insights on the Temperature of the whole of Singapore and each postal area for the past 20 years.
  3. Insights on the relationship of the Rainfall Precipitation and Temperature in the different months/seasons yearly.

Target Group:

  • General Public, people living in Singapore

DATASET

The Data Sets we will be using for our analysis and for our application is listed below:

Data/Source Variables/Description Rationale Of Usage

Temperature and Rainfall Data
(Jan 2012 - Dec 2019)

(http://www.weather.gov.sg/climate-historical-daily/)

  • Stations
  • Date
  • Daily Rainfall
  • Highest 30-min/60-min/120-min Rainfall (mm)
  • Mean/Minimum/Maximum Temperature (°C)
  • Mean/Max Wind Speed (km/h)

This dataset covers a good time series of Singapore's weather from 2012 to 2019 across different weather categories. Our team wish to spot the trend or pattern of Singapore's climate in every town if possible.

Amenities Location Data

(https://api.data.gov.sg/v1/environment/rainfall) (https://api.data.gov.sg/v1/environment/air-temperature)

  • Station ID
  • Station Name
  • Latitude
  • Longitude

The data set will be used to anchor the amenities available for the selected property in a specified range

BACKGROUND SURVEY OF RELATED WORK

Below are a few visualizations and charts we considered making for our projects.

Visual Considerations Insights / Comments

Title: Qualitative Thematic Map
ThematicMap.png

Source: https://mapdesign.icaci.org/2014/12/mapcarte-353365-life-in-los-angeles-by-eugene-turner-1977/

One of the items that we looked at is this qualitative thematic map that was covered in class.

From our initial brainstorming of ideas, we intend to look at various factors that a buyer will look at, giving the buyer a high-level overview of the different areas and whether it fits the criteria that he chooses. How we will adapt ideas from this graph is for us to allow the users to make a few selections of multiple factors. Then based on which criteria the different properties in the different subzones can meet, we can choose different shapes, colours to represent the zone.



Title: Whisker plot of temperature
Temperature whisker.png

Source: https://www.ck12.org/statistics/box-and-whisker-plots/rwa/The-Ways-of-Weather/

We can see the temperature for the selected area over a year. The whisker plots can show the upper and lower boundaries of temperature, and we can observe that the temperature gradually rises to a peak from Jan to Aug, before decreasing until December.

We hope to apply this chart to display the rainfall for a selected area over a year. This allows buyers to be able to better understand the rainfall pattern in the area so that he can better understand if the area suits his preferences.


Title: Heatmap of rainfall
Heatmap of rainfall.png

Source: https://www.shanelynn.ie/analysis-of-weather-data-using-pandas-python-and-seaborn/

This is a heatmap of daily rainfall. Darker colours of red represent heavier rainfall.

Another way to have a visualization to understand the patterns of rainfall. Through this, we can quickly see how many days in a year where there is rain for a selected subzone. Assuming that a potential buyer is interested in a property that sees more sunlight, he will be more interested in a subzone where the graph looks brighter. On the other hand, if the buyer is interested in a property that is always rainy, he would be interested in an area where the graph looks darker.


Title: Spatial Interpolation
Property heatmap.png

Source: https://www.srx.com.sg/heat-map/

This graph shows the property prices in Singapore for different property types. The user can choose to select different property types, and the graph will update to show only the selected property type. A variety of colours are chosen here to display different levels of prices.

Based on our problem, there are 2 key aspects that we are looking at: Prices and Weather. One way this kind of visualization could be utilized by our group is for us to use this to display prices or Weather in Singapore across all subzones. By charting either prices or the various weather types over a map of Singapore, the user will be able to quickly gain an understanding of how the different criteria that he can choose will be like across Singapore.



KEY TECHNICAL CHALLENGES & MITIGATION

No. Challenge Description Mitigation Plan
1.
Software Challenge Unfamiliarity of visualisation tools such as R, R Shiny, Tableau.
  • Github Learning
  • Stackoverflow research
  • Self-directed and peer learning
  • Watch video tutorials from YouTube
  • Hands-on practice using the different training platforms such as Data Camps
2.
Programming Challenge Inexperince with data cleaning and transformation using R
  • Trial and error
  • Read online articles and forums for guidance
  • Watch video tutorials on how to fully utilise packages such as lapply, tidyr and dplyr
3.
Workload Constraint Time and Workload Constrains
  • Design reasonable project timeline based on everyone's ability and capacity.
  • Set milestones and adjust the timeline accordingly based on the team's progress.
4.
Dataset Complexity

Our have different data from multiple sources in multiple different formats, hence we foresee a huge challenge in standardizing the data

  • Note: Our current dataset is looking at 49 areas over the spread of 8 years of data, for every year there are 12 months of data. This gives a total of 4,704 CSV files to consolidate and clean for weather data alone.
  • Make use of data preparation tools such as tableau prep
  • Make use of our database management skills to normalize all data tables into third normal form

STORYBOARD

Dashboards Description

Dashboard 1: Qualitative Thematic Map of Singapore property
VA1.jpg

Our group plans to do an interactive Qualitative Thematic Map which reflects different faces based on the year, price, weather, amenities filters adjusted by the users. This chart will show the data at a high level for users to identify which area meets their needs in a glance. Users can also view the map based on the postal area or zone.

Filters used includes:

  • Sliders
  1. Year
  2. Transacted Price
  3. Amenities
  • Single Dropdown List
  1. Weather
  2. Map level of detail
  • Multiple Drowndown List
  1. Property Type


Based on User’s adjustment for the filters, the map would reflect the user 3 different data types:

  • Weather
When the postal area/zone’s selected weather is above the median of Singapore’s data over the selected year, the chart will reflect a circle face shape. However, if the postal area/zone’s selected weather falls in the median or is lower than the median of Singapore’s data over the selected year, the chart will reflect a circle face shape with devil horns.
  • Pricing
When the postal area/zones have houses with pricing over the selected year that meets the requirements of the user based on his/her filter, the chart will reflect a smiling face. However, if the postal area/zones have houses with pricing over the selected year that is lower than the requirements of the user based on his/her filter, the chart will reflect a blank face. Lastly, if the postal area/zones have houses with pricing over the selected year that is more than the requirements of the user based on his/her filter, the chart will reflect a sad face.
  • Amenities
When the postal area/zone’s number of amenities in the selected year meets the requirements of the user based on the filter, the chart will reflect the faces to be shown in green colour. However, if the postal area/zone’s number of amenities does not meet the requirements of the user, the chart will reflect the faces in blue colour.


With this visualisation, users will be able to identify and shortlist the area(s) that meets their requirements the best.


Dashboard 2: Property prices in Postal Areas{Based on Users selected areas in storyboard 1}
VA2.jpg

The purpose of this chart is to show a detailed breakdown of the properties that meet the requirements of the Users based on his/her filters and User's shortlisted area(s) in the chart from Dashboard 1.


This chart shows all the properties and their prices for all the properties that meet the requirements of the Users based on the filter’s range and the shortlisted area(s).


Filters used includes:

  • Sliders
  1. Year
  2. Transaction Price
  3. Amenities
  • Single Dropdown List
  1. Property Type
  2. Sorted By


The chart will be able to help users better understand the property prices based on his/her shortlisted area(s) and make a decision on which area’s property to purchase.


X-Axis: Transacted Properties’ Name
Y-Axis: Transacted Pricing


This chart will be shown together with the chart on Dashboard 3 to help the buyer make the best-informed decisions.


Dashboard 3: Distribution of Rain Precipitation Amount/ Temperature/ Wind Speed in Postal Areas{Based on Users selected areas in Dashboard 1}
VA3.jpg

The purpose of this chart is to show a detailed breakdown of the weather (which includes Rain Precipitation Amount/ Temperature/ Wind Speed) for User’s shortlisted area(s) from the chart in Dashboard 1.


Filters used includes:

  • Sliders
  1. Year
  2. Month
  • Single Dropdown List
  1. Level of Detail


The user can adjust the filters to identify any patterns or trends of the weather based on the short-listed areas in Singapore. This chart can help the user better identify which area best suits the user based on his preferences and needs.


X-Axis: Level of Detail(Sub-zone/ Postal Area/ Zone)
Y-Axis: Rain Precipitation Amount/ Temperature/ Wind Speed


This chart will be shown together with the chart on Dashboard 2 to help the buyer make the best-informed decisions.


Dashboard 4: Comparing Rainfall {selected weather} and the median pricing of All Properties{selected property type}
VA4.jpg

The chart in this storyboard reflects a combination of two thematic maps, a bar chart map as well as a spatial interpolation map. Similar to the chart in Dashboard 1, this chart will show data at a high level for users to identify which area meets their needs in a glance. However, this chart is more-straightforward as the results shown in this chart is more clear cut and less informative.


In this chart, the user will be able to compare:

  • The weather of the user’s choice in each zone/postal area.
  • Median pricing of their selected property type in each zone/postal area.


Filters used includes:

  • Sliders
  1. Year
  • Single Dropdown List
  1. Level of Detail
  2. Weather
  3. Property Type


With the visualization, users will be able to compare all the areas in Singapore to make an informed decision. By doing so, the chart can help the user in shortlisting the area that they wish to zoom into in another view of light as compared to the chart in Dashboard 1 where detailed comparison of the whole Singapore is limited.

MILESTONES

Photo 2020-03-01 16-22-33.jpg

COMMENTS

No. Name Date Comments
1. (Name) (Date) (Comment)
2. (Name) (Date) (Comment)
3. (Name) (Date) (Comment)