Difference between revisions of "Group07 Proposal"

From Visual Analytics and Applications
Jump to navigation Jump to search
Line 23: Line 23:
 
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Background</font></div>
 
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Background</font></div>
  
E-commerce represented 54% of U.S. sales in the third quarter for TV and web merchant QVC.com. QVC is one of the most successful TV shopping broadcasters worldwide, which is one of eight leading retail brands under the Qurate Retail Group, focuses on building customer relationship, to engage with them to discover a dynamic catalogue of products from a wide range of categories. QVC is committed to providing its customers with thousands of the most innovative and contemporary beauty, fashion, jewellery and home products. Customers are engaged via multiple channels, through social media like Facebook Live, proprietary platforms that focuses on specific product types and video commerce based on QVC’s television channel.
 
  
<br>
+
The Singapore Real Estate market has always been a hot topic for years ever since 1995. Government, local banks, developers and investors from all over the world has been closely watching Singapore property market. In the last 20+ years, the Singapore property price witnessed a roller coaster of change. [https://www.ura.gov.sg/Corporate/Media-Room/Media-Releases/pr18-40 URA-Media Releases] [https://www.ura.gov.sg/-/media/Corporate/Media-Room/2018/Jul/pr18-40b.pdf Appendix]. In the most recent 10 years, since Singapore economy rebounded back after 2008 financial crisis. As a result, Singapore Real Estate market price has been heating up significantly from 2009 onwards, until Singapore Government announced that the property tax rates will be made more progressive over two years from Jan 2014, the price had been cooled down.[https://www.rikvin.com/taxation/singapore-property-tax-information/ 2014-2015 Singapore property tax changes]
  
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Motivation</font></div>
+
After a four-year slump until the end of 2017, the property market started to bounce back, and it has the trend that will go up even more. On the other hand, “the US Federal Reserve’s interest-rate has been increased 5 times since President Trump took office in Jan 2017”, says from Bloomberg news, “and there will be 2 more increase happen in 2018”. [https://www.bloomberg.com/news/articles/2018-07-19/trump-trespasses-on-fed-independence-blasting-powell-rate-hikes?utm_campaign=socialflow-organic&utm_content=business&utm_medium=social&cmpid=socialflow-facebook-business&utm_source=facebook Trump Blasts Powell’s Rate Hikes, Trespassing on Fed’s Independence] This foreseeing news will have impact on Singapore economy in near future. How does this news will affect Singapore residential property price? As Singapore IRAS (Inland Revenue Authority of Singapore) made an announcement this February, says that the property tax will increase to 4% for property house price over 1 million SGD.  [https://www.iras.gov.sg/irashome/News-and-Events/Singapore-Budget/Budget-2018---Overview-of-Tax-Changes/ Budget 2018-Overview tax changes]  Officials state that this new announcement is not a cooling measure. [https://www.straitstimes.com/singapore/higher-stamp-duty-not-a-cooling-measure-analysts Higher stamp duty not a cooling measure, say analysts]. 
  
To conduct a better business strategy for e-commerce from unsupervised massive transaction data is extremely challenging. The best way to turn analytics, metrics, and raw data into a universal story that everyone can get on board with is through data visualization. With that, we can understand the impacts of a supply chain design change on service, sustainability and risk, as well as to understand our customer better. It unveils the abnormal pattern from the operational perspective, company would easily know where the pain points locate so as to make improvements, not only for the profitability of company itself, but also to provide customers with a more pleasant shopping experience. Because shopping is enjoying!
+
But is it really not a cooling measure? Or is it to protect domestic banks from the developers that might potentially being unable to pay back loans after the Federal Reserve’s interest-rate go up?
  
 
<br>
 
<br>
Line 35: Line 34:
 
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Objectives</font></div>
 
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Objectives</font></div>
  
The data from QVC, a giant Home-Shopping company, comprises millions of transactions and thousands of physical locations, traditional statistics figure can only tell partial story, our group decides to
 
* visualize the value behind data from a single Web-enabled tool
 
* reveal consumer behaviour patterns for the days of the week
 
* forecast the demand of certain products of the region from inventory management perspective.
 
  
We aim to generate a more efficient streamline upon current supply chain and provide an effortless too to help company make better decisions.
+
There is a deficiency of Market Watch tool for analysing the real estate market data as well as the trends properly. As it is very important to show the trendline cross different years and time, however, these data are still in table formats and the visualizations are quite basic, most of trends are illustrated statically, static graphs are not explanatory enough to show a full picture of changes, which sets barriers for readers from getting any useful insights and findings.
 +
To better understand how the Real Estate market price moves cross different type of sales, property types and planning areas in the last 20+ years. And to answer the questions, for instances:  How does Singapore Property Market pattern changes overtime after 2008-2012 heat-up? Does the government make action towards the market heat-up effectively? How does market respond when the policies released? A tool that is capable to compare the market trends in different time period would be desirable.
 +
Our team aims to build such a web visual analytics-driven application as a Market Watch tool for readers to understand and compare the Property Market pattern changing overtime.  
 +
 
  
 
<br>
 
<br>
Line 46: Line 44:
 
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Scope</font></div>
 
<div style="border-style: solid; border-width:0; background: #0099ff; padding: 7px; font-weight: bold; text-align:left; line-height: wrap_content; text-indent: 20px; font-size:20px; font-family:Century Gothic;border-bottom:5px solid white; border-top:5px solid black"><font color= #ffffff>Project Scope</font></div>
  
Our project consists of the following components:
+
Our project consists of the following components :
 +
==NEED TO CHANGE==
 
# <b>Data cleaning and preparation:</b> We need to remove invalid records from the dataset. We also need to prepare the dataset in a way that allows the plotting of network graphs. This enables users to see the flow of goods from warehouses to consumers, and could be helpful for distribution management.
 
# <b>Data cleaning and preparation:</b> We need to remove invalid records from the dataset. We also need to prepare the dataset in a way that allows the plotting of network graphs. This enables users to see the flow of goods from warehouses to consumers, and could be helpful for distribution management.
# <b>Time series analysis and forecasting:</b> Using historical data, we will identify if there is any seasonality in customer orders. Then, we will apply time-series forecasting methods to provide users an estimated demand figure for future periods, which could be helpful for inventory management and procurement planning.
+
# <b>Time series analysis:</b> Using historical data, we will identify if there is any seasonality in customer orders. Then, we will apply time-series forecasting methods to provide users an estimated demand figure for future periods, which could be helpful for inventory management and procurement planning.
 
# <b>Calendar plot:</b> We will visualize the number of customer orders using a calendar plot. This enables users to see which months and days of week have higher demand, and this information could help them in manpower management. For example, if I know there are more demands coming in on weekends close to year-end, I can hire more temporary workers to process and deliver the orders.
 
# <b>Calendar plot:</b> We will visualize the number of customer orders using a calendar plot. This enables users to see which months and days of week have higher demand, and this information could help them in manpower management. For example, if I know there are more demands coming in on weekends close to year-end, I can hire more temporary workers to process and deliver the orders.
 
# <b>Warehouse performance analysis:</b> We will calculate and pinpoint the warehouses which have high proportions of delivery delays, and uncover possible reasons behind the delay. (e.g. certain product categories have more delayed deliveries; long distances between warehouse and consumer)
 
# <b>Warehouse performance analysis:</b> We will calculate and pinpoint the warehouses which have high proportions of delivery delays, and uncover possible reasons behind the delay. (e.g. certain product categories have more delayed deliveries; long distances between warehouse and consumer)
Line 71: Line 70:
 
|-
 
|-
 
|
 
|
Sales_Order_Nbr
+
No. of Units
 
||   
 
||   
Unique identifier for a given transaction
+
Number of units sold for each transaction
 
|-
 
|-
 
|
 
|
Sales_Order_Line_Nbr
+
Area (sqm)
 
||
 
||
Unique sub-identifier for a given transaction
+
Spacial measure of the unit sold
 
|-
 
|-
 
|  
 
|  
Package_Id
+
Type of Area
 
||
 
||
ID code used to identify a specific box or package
+
Type of area is either Strata or Land
 
|-
 
|-
 
|  
 
|  
Order_Dt
+
Transacted Price ($)
 
||
 
||
Date the transaction was placed
+
Amount of each transaction ($)
 
|-
 
|-
 
|  
 
|  
Party_Id
+
Unit Price ($ psm)
 
||
 
||
Unique identifier for an individual or organization
+
Price per Square Meter
 
|-
 
|-
 
|  
 
|  
Order_Type_Cd
+
Unit Price ($ psf)
 
||   
 
||   
Code used to the type of Order
+
Price per Square Feet
 
|-
 
|-
 
|  
 
|  
Shipping_Priority_Ind
+
Sale Date
 
||   
 
||   
Indicator used to indicate if order will be fulfilled with prioritized shipping (e.g. Overnight)
+
Date of Sales settled
 
|-
 
|-
 
|  
 
|  
Total_Line_Amt
+
Property Type
 
||   
 
||   
Total amount owed for the specific order line
+
Types can be: Apartment/Condominium/Executive Condominium/Detached House/Semi-Detached House/Terrace House
 
|-
 
|-
 
|
 
|
Unit_Price_Amt
+
Tenure
 
||   
 
||   
Merchandise price for the specific line
+
A duration of holding an interest in land or other such real property
 
|-
 
|-
 
|
 
|
Line_Status_Cd
+
Completion Date
 
||   
 
||   
Status of the specific order line (refer to status reference sheet)
+
Property completed date
 
|-
 
|-
 
|
 
|
Line_Status_Dt
+
Type of Sale
 
||   
 
||   
Date the specific order line was set to the current status
+
New Sale/Resale/Sub Sale
 
|-
 
|-
 
|
 
|
Product_Id
+
Purchaser Address Indicator
 
||   
 
||   
Unique identifier for an item presented for purchase (e.g. The number seen on TV)
+
HDB / Private
 
|-
 
|-
 
|
 
|
Skn_Id
+
Postal District
 
||   
 
||   
Internal ID used for back office purposes
+
Two digits of Singapore postal district
 
|-
 
|-
 
|
 
|
Sku_Id
+
Postal Sector
 
||   
 
||   
Unique ID used to denote a Product's specifics (e.g. specific color and size)
+
1st two digits of 6-digit postal codes.  
 
|-
 
|-
 
|
 
|
Color_Desc
+
Postal Code
 
||   
 
||   
Description of the color
+
Detail location of the property
 
|-
 
|-
 
|
 
|
Size_Desc
+
Planning Region
 
||   
 
||   
Description of the size
+
To facilitate urban planning, the Urban Redevelopment Authority (URA) divides Singapore into regions, planning areas and subzones.
 
|-
 
|-
 
|
 
|
Shipped_Dt
+
Planning Area
 
||   
 
||   
Date the order line was shipped (left QVC's warehouse)
+
To facilitate urban planning, the Urban Redevelopment Authority (URA) divides Singapore into regions, planning areas and subzones.
|-
 
|
 
Source_Ship_Warehouse_Nbr
 
|| 
 
Unique identifier denoting the warehouse where the product is shipped from
 
|-
 
|
 
Assigned_Dc_Id
 
||
 
Distribution center assigned to order at time of creation
 
|-
 
|
 
Cancelled_Qty
 
||
 
Number of units cancelled
 
|-
 
|
 
Ordered_Qty
 
||
 
Number of units requested by customer
 
|-
 
|
 
Shipped_Qty
 
||
 
Number of units shipped to customer
 
|-
 
|
 
Merchandise_Div_Desc
 
||
 
Description of Merchandise Division
 
|-
 
|
 
Merchandise_Dept_Desc
 
||
 
Description of Merchandise Department
 
|-
 
|
 
Carrier_Used_Tracking_Id
 
||
 
ID used to denote a unique package shipment
 
|-
 
|
 
Shipment_Status_Dt
 
||
 
The most recent date of the shipment's status
 
|-
 
|
 
Pickup_Dt
 
||
 
Date the package was picked up by the carrier from QVC
 
|-
 
|
 
Scheduled_Delivery_Dt
 
||
 
Date the package is believed to arrive
 
|-
 
|
 
Rescheduled_Delivery_Dt
 
||
 
Revised date the package is believed to arrive
 
|-
 
|
 
Package_Scan_Dttm
 
||
 
Date the last time the package was scanned along its journey
 
|-
 
|
 
Package_Cnt
 
||
 
Number of packages associated with a specific order line
 
|-
 
|
 
Actual_Total_Package_Qty
 
||
 
Number of packages associated with a specific order line (post any consolidation)
 
|-
 
|
 
Delivery_Confirmation_Dt
 
||
 
Date the package was delivered to the consumer
 
|-
 
|
 
SHIP_TO_CITY
 
||
 
City the package was delivered to
 
|-
 
|
 
SHIP_TO_STATE
 
||
 
State the package was delivered to
 
|-
 
|
 
SHIP_TO_ZIP
 
||
 
Zip Code the package was delivered to
 
 
|-
 
|-
 
|}
 
|}

Revision as of 20:23, 21 July 2018

Overview

Proposal

Report

Poster

Application

All Projects


Project Background


The Singapore Real Estate market has always been a hot topic for years ever since 1995. Government, local banks, developers and investors from all over the world has been closely watching Singapore property market. In the last 20+ years, the Singapore property price witnessed a roller coaster of change. URA-Media Releases Appendix. In the most recent 10 years, since Singapore economy rebounded back after 2008 financial crisis. As a result, Singapore Real Estate market price has been heating up significantly from 2009 onwards, until Singapore Government announced that the property tax rates will be made more progressive over two years from Jan 2014, the price had been cooled down.2014-2015 Singapore property tax changes

After a four-year slump until the end of 2017, the property market started to bounce back, and it has the trend that will go up even more. On the other hand, “the US Federal Reserve’s interest-rate has been increased 5 times since President Trump took office in Jan 2017”, says from Bloomberg news, “and there will be 2 more increase happen in 2018”. Trump Blasts Powell’s Rate Hikes, Trespassing on Fed’s Independence This foreseeing news will have impact on Singapore economy in near future. How does this news will affect Singapore residential property price? As Singapore IRAS (Inland Revenue Authority of Singapore) made an announcement this February, says that the property tax will increase to 4% for property house price over 1 million SGD. Budget 2018-Overview tax changes Officials state that this new announcement is not a cooling measure. Higher stamp duty not a cooling measure, say analysts.

But is it really not a cooling measure? Or is it to protect domestic banks from the developers that might potentially being unable to pay back loans after the Federal Reserve’s interest-rate go up?


Project Objectives


There is a deficiency of Market Watch tool for analysing the real estate market data as well as the trends properly. As it is very important to show the trendline cross different years and time, however, these data are still in table formats and the visualizations are quite basic, most of trends are illustrated statically, static graphs are not explanatory enough to show a full picture of changes, which sets barriers for readers from getting any useful insights and findings. To better understand how the Real Estate market price moves cross different type of sales, property types and planning areas in the last 20+ years. And to answer the questions, for instances: How does Singapore Property Market pattern changes overtime after 2008-2012 heat-up? Does the government make action towards the market heat-up effectively? How does market respond when the policies released? A tool that is capable to compare the market trends in different time period would be desirable. Our team aims to build such a web visual analytics-driven application as a Market Watch tool for readers to understand and compare the Property Market pattern changing overtime.



Project Scope

Our project consists of the following components :

NEED TO CHANGE

  1. Data cleaning and preparation: We need to remove invalid records from the dataset. We also need to prepare the dataset in a way that allows the plotting of network graphs. This enables users to see the flow of goods from warehouses to consumers, and could be helpful for distribution management.
  2. Time series analysis: Using historical data, we will identify if there is any seasonality in customer orders. Then, we will apply time-series forecasting methods to provide users an estimated demand figure for future periods, which could be helpful for inventory management and procurement planning.
  3. Calendar plot: We will visualize the number of customer orders using a calendar plot. This enables users to see which months and days of week have higher demand, and this information could help them in manpower management. For example, if I know there are more demands coming in on weekends close to year-end, I can hire more temporary workers to process and deliver the orders.
  4. Warehouse performance analysis: We will calculate and pinpoint the warehouses which have high proportions of delivery delays, and uncover possible reasons behind the delay. (e.g. certain product categories have more delayed deliveries; long distances between warehouse and consumer)
  5. Distribution analysis: We will visualize if there are any geographical gaps between warehouse supply and consumer demand, which could help users decide if they should rearrange their stocks to more strategic locations.
  6. Geospatial analysis: We may uncover spatial patterns in consumer demand, which can be helpful for distribution optimization by stocking spatially-correlated product categories or SKUs in warehouses of the same area.



Dataset Overview

Datasets are extracted from http://ibit.temple.edu/analytics/can-small-independent-pharmacies-compete-with-the-big-chains/
All 6 data sets with 36 columns were in CSV format, total number of rows after joining data sets - 4,680,635 rows. The final dataset covers 6 months time period [July 2016 - Dec 2016] and geographically represents U.S. territory.

Metadata

Variable

Description

No. of Units

Number of units sold for each transaction

Area (sqm)

Spacial measure of the unit sold

Type of Area

Type of area is either Strata or Land

Transacted Price ($)

Amount of each transaction ($)

Unit Price ($ psm)

Price per Square Meter

Unit Price ($ psf)

Price per Square Feet

Sale Date

Date of Sales settled

Property Type

Types can be: Apartment/Condominium/Executive Condominium/Detached House/Semi-Detached House/Terrace House

Tenure

A duration of holding an interest in land or other such real property

Completion Date

Property completed date

Type of Sale

New Sale/Resale/Sub Sale

Purchaser Address Indicator

HDB / Private

Postal District

Two digits of Singapore postal district

Postal Sector

1st two digits of 6-digit postal codes.

Postal Code

Detail location of the property

Planning Region

To facilitate urban planning, the Urban Redevelopment Authority (URA) divides Singapore into regions, planning areas and subzones.

Planning Area

To facilitate urban planning, the Urban Redevelopment Authority (URA) divides Singapore into regions, planning areas and subzones.



Deliverables

At the end of the project, we will submit the following deliverables:

  • R Shiny web application
  • Research paper
  • Poster
  • Project artifacts


Schedule
Group7 AY1718T3 Schedule.jpeg