Difference between revisions of "S-MALL Overview"

From Visual Analytics and Applications
Jump to navigation Jump to search
 
(42 intermediate revisions by 3 users not shown)
Line 4: Line 4:
 
<p><font size = 3; color="#FFFFFF"><span style="font-family:Century Gothic;">Team S-MALL: Chen Yun-Chen | Chiam Zhan Peng | Zheng Bijun</span></font></p>
 
<p><font size = 3; color="#FFFFFF"><span style="font-family:Century Gothic;">Team S-MALL: Chen Yun-Chen | Chiam Zhan Peng | Zheng Bijun</span></font></p>
 
</div>
 
</div>
 +
 
{|style="background-color:#565656;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0"  |  
 
{|style="background-color:#565656;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0"  |  
| style="font-family:Century Gothic; font-size:100%; solid #000000; background:#002060; text-align:center;" width="20%" |   
+
| style="font-family:Century Gothic; font-size:100%; solid #000000; background:#2e5b84; text-align:center;" width="25%" |   
 
;  
 
;  
 
[[S-MALL_Overview| <font color="#FFFFF">Overview</font>]]  
 
[[S-MALL_Overview| <font color="#FFFFF">Overview</font>]]  
+
 
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="20%" |   
+
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="25%" |   
 
;  
 
;  
[[S-MALL_Proposal| <font color="#FFFFFF">Proposal</font>]]  
+
[[S-MALL_Application| <font color="#FFFFFF">Application</font>]]  
+
 
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="20%" |   
+
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="25%" |   
 
;  
 
;  
[[S-MALL_Application| <font color="#FFFFFF">Application</font>]]  
+
[[S-MALL_User Guide| <font color="#FFFFFF">User Guide</font>]]  
 
   
 
   
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="20%" |   
+
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="25%" |   
 
;  
 
;  
 
[[S-MALL_Poster| <font color="#FFFFFF">Poster</font>]]  
 
[[S-MALL_Poster| <font color="#FFFFFF">Poster</font>]]  
 
| style="font-family:Century Gothic; font-size:100%; solid #1B338F; background:#565656; text-align:center;" width="20%" | 
 
;
 
[[S-MALL_Reports| <font color="#FFFFFF">Reports</font>]]
 
 
   
 
   
 
|  &nbsp;  
 
|  &nbsp;  
Line 31: Line 28:
 
== ABSTRACT ==
 
== ABSTRACT ==
  
<p align="justify"><font size = 2><span style="font-family:Century Gothic;">With growing popularity of e-commerce and online shopping, traditional brick & mortar retail malls are facing stiff challenge and need to reinvent itself and compete with these new “online” channels. As part of the Smart-Nation drive and transformation, retail malls can leverage on this digital transformation journey to find its own unique value preposition with its physical and “offline” presence. With new technologies and connected era like IoT, shoppers are leaving their digital footprints and trackable just like on-line customers.</span></font></p>
+
<p align="justify">With growing popularity of e-commerce and online shopping, traditional brick & mortar retail malls are facing stiff challenge and need to reinvent itself and compete with these new “online” channels. As part of the Smart-Nation drive and transformation, retail malls can leverage on this digital transformation journey to find its own unique value proposition with its physical and “offline” presence. With new technologies and connected era like IoT, shoppers are leaving their digital footprints and trackable just like on-line customers.</p>
 +
 
 +
<p align="justify">Retail malls have data such as presence and movement via Wi-Fi access point with customers’ mobile devices, traditional transaction data gathered from daily operations and customers profile data obtained from loyalty programs. The opportunity is to discover patterns and relationship within the data and offer deeper insights, formulate marketing strategies for retail stores and better experience for their customers. This project aims to design and develop a web-based application that provides such analytical visualization. It is developed using open-source R Shiny framework and several R packages such as ggplot2, chorddiag, hexbin, sunburstR, highcharter, arules, visNetwork.</p>
 +
 
 +
<p align="justify">The motivation and objectives will be discussed followed by detailed discussion of the principles, approach and data visualization techniques that are used. Using actual data from a well-known shopping mall, we will demonstrate the functionality of the application in visualizing and discovering the patterns such as peak hour, busy area, movement and customers behavior associated with their profile and transactions. Finally, we will conclude by providing some insights and potential recommendations for their mall operations and strategy.</p>
  
<p align="justify"><font size = 2><span style="font-family:Century Gothic;">Retail malls have data such as presence and movement via Wi-Fi access point with customers’ mobile devices, traditional transaction data gathered from daily operations and customers profile data obtained from loyalty programs. The opportunity is to discover patterns and relationship within the data and offer deeper insights, formulate marketing strategies for retail stores and better experience for their customers. This project aims to design and develop a web-based application that provides such analytical visualization. It is developed using open-source R Shiny framework and several R packages such as ggplot2, chorddiag, hexbin, sunburstR, highcharter, arules, visNetwork.</span></font></p>
+
== MOTIVATION ==
  
<p align="justify"><font size = 2><span style="font-family:Century Gothic;">The motivation and objectives will be discussed followed by detailed discussion of the principles, approach and data visualization techniques that are used. Using actual data from a well-known shopping mall, we will demonstrate the functionality of the application in visualizing and discovering the patterns such as peak hour, busy area, movement and customers behavior associated with their profile and transactions. Finally, we will conclude by providing some insights and potential recommendations for their mall operations and strategy.</span></font></p>
+
[[File:T3 Motivation.png|center|1200px]]
  
 +
<p align="justify">In view of the competitions from online e-commerce channels, there are stronger interests and motivation for key stakeholders in the physical retail malls space to better understand their operations and business so as to provide better value and experience for their customers.  We will analyze from a Who, What, Why, How approach to better understand in details and frame the objectives of this project.</p>
  
== MOTIVATION ==
+
<p align="justify">WHO – There are 3 key stakeholders involved in the retail shopping mall. They are the mall management, retailers and customers. Mall management are the owner of the mall and are keen to ensure all the rental space are occupied, maximize the rental revenue and able to attract customers to visit the mall on regularly basis. For the retailers, they are keen to know the amount, pattern of footfall and profile of customers that visits their store to better optimize their operating and marketing campaigns. For customers, they would like to have better personalized service which will enhance their shopping experience.
 +
</p>
  
[[File:T3 Motivation.png|center|1000px]]
+
<p align="justify">WHAT – This focuses on the key problems and current issues. Even with abundance of data or data accessibility, they are not used effectively. There is no visibility of the footfall, traffic density or journey movement of the customers. The limited mall space is not designed to be best used and rental rate are based on qualitative estimates rather than quantitative parameter to differentiate their rental rates. There are poor insights of customers profile, dwell duration and transactions and this stops the mall and retailers from providing better or differentiate and personalized experience for the customers.
 +
</p>
  
<p align="justify"><font size = 2><span style="font-family:Century Gothic;">add motivation here</span></font></p>
+
<p align="justify">WHY – This focuses on the reason on why it matters. Understanding traffic intensity and flow, dwell duration helps understand the patterns of the customers. Rental rates can be based on facts of footfall patterns and trends rather than fixed based on floor or better location. Linking the movement with the profile and transactions of customers allows for customer segmentation with targeted marketing campaigns. 
 +
HOW – This will identify on what is required to support the key usage scenarios for the key stakeholders. The area density, inter-floor movement will be the first key business objective followed by the journey, duration and profile associated with it. This will support retail management and retailers to better optimize their operations and marketing campaigns and activities. Association rules analysis will further allow link and drill down into transaction behavior, product-mix and market-basket analysis. 
 +
</p>
  
 +
== PRINCIPLE & METHODOLOGY ==
 +
<table>
 +
<tr>
 +
<td width=45%>[[File:T3 principal.png|center|600px]]</td>
 +
<td width=3%></td>
 +
<td width=45%>
 +
==== Key objectives ====
  
<div style="text-align:left; padding-top:5px; font-family:Calibri;">
+
<li>Address challenge of data accessibility</li>
<font size = 5>Data Description</font>
+
<li>Combine and link silos dataset</li>
[[File:T3 Datades.PNG|center|800x300 px]]
+
<li>Reproducible application for business user consumption</li>
<li>Profile Data: contains demographic information of the shopping mall members</li>
 
<li>Transaction Data: contains two months transnational records from January to February 2017</li>
 
<li>Wi-Fi Sensor Data: contains two months Wi-Fi sensor records captured in the mall, which can be used to analyze the movement of customers</li>
 
<li>Maps: shopping mall layouts</li>
 
</div>
 
  
<div style="text-align:left; padding-top:5px; font-family:Calibri;">
+
</td>
<font size = 5>Expected Outcome</font>
+
</tr>
<li>Data Integration: Combine the three dataset to derive patterns, associations and actionable insights</li>
+
</table>
<li>Interactive Visualization:
 
<ol>
 
<li>Overview of customers movements patterns by weeks, days, hours</li>
 
<li>Inter-floor movement</li>
 
<li>Customers' profile and transactions integration </li>
 
</ol>
 
<li>Use Cases: Zone Traffic Statistics, Zone Traffic Flow, Staff Planning, Marketing/Event Campaign</li>
 
</div>  
 
  
<div style="text-align:left; padding-top:5px; font-family:Calibri;">
+
== DATA DESCRIPTION ==
<font size = 5>Visualization Tool & Packages</font>
+
[[File:T3 Datades.PNG|center|600x300 px]]<br><br>
<li>R: Hexagonal binning using ggplot2 and Kernel Decimal Estimate using stat_Density2d</li>
+
[[File:T3 Dataview.png|center|1000px]]
<li>R: Chord diagram using chorddiag</li>
 
<li>R: Shinyapp, shinydashboard, flexdashboard</li>
 
</div>
 
  
== Web-Based Visualization Application==
+
<div style="text-align:left; padding-top:5px; ">
Part1:
+
==== Data Preparation Tasks ====
{| class="wikitable"
 
|-
 
! scope="col" style="width: 20%;"| Visualization
 
! scope="col" style="width: 30%;"| Methodology & Technique
 
! scope="col" style="width: 30%;"| Usage
 
|-
 
! scope="row"| insert image of line chart
 
|
 
<ul>
 
<li>Chart type: Line chart & trellis plot</li>
 
<li>R Package: ggplot2, plotly</li>
 
<li>Interactivity: Use selectInput to control plot and segment by different timelevels, eg. Date, day of week, and hour</li>
 
</ul>
 
|
 
<ul>
 
<li>Line chart without trellis (segment=None) can be used to analyze the daily/weekly/hourly pattern of footfalls.</li>
 
<li>Trellis plot can be used to detect the cycling pattern over time.</li>
 
</ul>
 
|-
 
! scope="row"| insert image of chord diagram
 
|
 
<ul>
 
<li>Chart type: Chord diagram</li>
 
<li>R Package: chorddiag</li>
 
<li>Interactivity: Set specific datetime using selectIput and sliderInput to view the traffic transfer across floors.<br>
 
Hover to each floor to see the destination of its outflow traffics.</li>
 
</ul>
 
|
 
<ul>
 
<li>Analyze traffic flow across floor for specific datetime selection</li>
 
</ul>
 
|-
 
! scope="row"| insert image of hexbin map
 
|
 
<ul>
 
<li>Chart type: Hexagonal binning map</li>
 
<li>R Package: hexbin, ggplot, plotly</li>
 
<li>Key parameter setting: number of bins is set to 50</li>
 
<li>Interactivity: Set specific datetime using selectIput and sliderInput to view the traffic density on each floor.<br>
 
Click on specific hexbin to investigate members identity.</li>
 
</ul>
 
|
 
<ul>
 
<li>Analyze traffic density of floors for specific datetime selection</li>
 
<li>Drill down to shoppers’ identity based on interested density area</li>
 
</ul>
 
|-
 
|}
 
  
Part2:
+
<li>Data sampling: temporary solution for computational challenge</li>
{| class="wikitable"
+
<li>Data cleaning: data types, demographic fields clean up</li>
|-
+
<li>Data joining: merge data from different sources</li>
! scope="col" style="width: 20%;"| Visualization
+
<li>Data transformation: matrix for chord diagram, path for sunburst diagram, transaction table for association analysis, node and edge table for network visualization</li>
! scope="col" style="width: 30%;"| Methodology & Technique
 
! scope="col" style="width: 30%;"| Usage
 
|-
 
! scope="row"| insert image of sunburst
 
|
 
<ul>
 
<li>Chart type: Sunburst diagram</li>
 
<li>R Package: sunburstR</li>
 
<li>Interactivity: Set radio button to view the journey of different member segments.<br>
 
Set minimum dwell time using sliderInput to exclude passing-by floors.<br>
 
Hover to see the path and relevant statistics.</li>
 
</ul>
 
|
 
<ul>
 
<li>Investigate popular shopping path of members based on floors.</li>
 
</ul>
 
|-
 
! scope="row"| insert image of treemap
 
|
 
<ul>
 
<li>Chart type: Treemap</li>
 
<li>R Package: treemap, highCharter </li>
 
<li>Interactivity: Set radio button to view the journey of different member segments.<br>
 
Click on floor level to drill down to store level.<br></li>
 
</ul>
 
|
 
<ul>
 
<li>Analyze members’ average dwell time on each floor and store.</li>
 
</ul>
 
|-
 
! scope="row"| insert image of boxploter
 
|
 
<ul>
 
<li>Chart type: Boxplot</li>
 
<li>R Package: plotly</li>
 
<li>Key parameter setting: number of bins is set to 50</li>
 
<li>Interactivity: Click on the treemap to get the relevant boxplot for selected floor.</li>
 
</ul>
 
|
 
<ul>
 
<li>Analyze dwell time distribution of each store based on floor selection on treemap.</li>
 
</ul>
 
|-
 
! scope="row"| insert image of bar chart
 
|
 
<ul>
 
<li>Chart type: Bar chart</li>
 
<li>R Package: plotly</li>
 
<li>Interactivity: Select on the boxplot distribution to view the demographic of interested members.<br>
 
Set the ‘profile count by’ parameter to decide the y-axis of bar plots.</li>
 
</ul>
 
|
 
<ul>
 
<li>Analyze members’ profile based on selection of their dwell time distribution. For example, we may analyze the high time spender of a specific store to see if they are of similar demographic.</li>
 
</ul>
 
|}
 
  
Part3:
+
==== R Packages for Data Preparation ====
{| class="wikitable"
+
[https://cran.r-project.org/web/packages/tidyverse/index.html tidyverse], [https://cran.r-project.org/web/packages/dplyr/index.html dplyr], [https://cran.r-project.org/web/packages/data.table/index.html data.table], [https://cran.r-project.org/web/packages/lubridate/index.html lubridate], [https://cran.r-project.org/web/packages/tidyr/index.html tidyr], [https://cran.r-project.org/web/packages/jpeg/index.html jpeg], [https://cran.r-project.org/web/packages/reshape2/index.html reshape2]
|-
+
</div>
! scope="col" style="width: 20%;"| Visualization
 
! scope="col" style="width: 30%;"| Methodology & Technique
 
! scope="col" style="width: 30%;"| Usage
 
|-
 
! scope="row"| insert image of bar
 
|
 
<ul>
 
<li>Chart type: Bar chart</li>
 
<li>R Package: ggplot, plotly</li>
 
<li>Interactivity: Set date range using radio button to see the plot of different months.</li>
 
</ul>
 
|
 
<ul>
 
<li>Investigate popular shopping path of members based on floors.</li>
 
</ul>
 
|-
 
! scope="row"| insert image of quadrant
 
|
 
<ul>
 
<li>Chart type: Scatter plot</li>
 
<li>R Package: arules, ggplot, plotly</li>
 
<li>Interactivity: Set date range using radio button to generate association rules based on different month’s transactions.<br>
 
Set parameters (support/confidence/min items) to generate valid associations rules and render plot. <br>
 
Hover over the bar to fade out non-relevant rules in the quadrant.<br>
 
Hover over the network to fade out non-relevant rules in the quadrant.</li>
 
</ul>
 
|
 
<ul>
 
<li>Analyze the competitive position of different rules based on set parameters.</li>
 
</ul>
 
|-
 
! scope="row"| insert image of network
 
|
 
<ul>
 
<li>Chart type: Network</li>
 
<li>R Package: visNetwork</li>
 
<li>Interactivity: Set date range using radio button to plot rules of different months.</li>
 
</ul>
 
|
 
<ul>
 
<li>Visualize the associations among departments.</li>
 
</ul>
 
|}
 

Latest revision as of 17:25, 14 August 2017

Turning Concrete Malls into Smart Malls (S-MALL):
A web-based analytics application for visualizing and mapping in-mall customer journeys and shopping behaviours

Team S-MALL: Chen Yun-Chen | Chiam Zhan Peng | Zheng Bijun

Overview

Application

User Guide

Poster

 


ABSTRACT

With growing popularity of e-commerce and online shopping, traditional brick & mortar retail malls are facing stiff challenge and need to reinvent itself and compete with these new “online” channels. As part of the Smart-Nation drive and transformation, retail malls can leverage on this digital transformation journey to find its own unique value proposition with its physical and “offline” presence. With new technologies and connected era like IoT, shoppers are leaving their digital footprints and trackable just like on-line customers.

Retail malls have data such as presence and movement via Wi-Fi access point with customers’ mobile devices, traditional transaction data gathered from daily operations and customers profile data obtained from loyalty programs. The opportunity is to discover patterns and relationship within the data and offer deeper insights, formulate marketing strategies for retail stores and better experience for their customers. This project aims to design and develop a web-based application that provides such analytical visualization. It is developed using open-source R Shiny framework and several R packages such as ggplot2, chorddiag, hexbin, sunburstR, highcharter, arules, visNetwork.

The motivation and objectives will be discussed followed by detailed discussion of the principles, approach and data visualization techniques that are used. Using actual data from a well-known shopping mall, we will demonstrate the functionality of the application in visualizing and discovering the patterns such as peak hour, busy area, movement and customers behavior associated with their profile and transactions. Finally, we will conclude by providing some insights and potential recommendations for their mall operations and strategy.

MOTIVATION

T3 Motivation.png

In view of the competitions from online e-commerce channels, there are stronger interests and motivation for key stakeholders in the physical retail malls space to better understand their operations and business so as to provide better value and experience for their customers. We will analyze from a Who, What, Why, How approach to better understand in details and frame the objectives of this project.

WHO – There are 3 key stakeholders involved in the retail shopping mall. They are the mall management, retailers and customers. Mall management are the owner of the mall and are keen to ensure all the rental space are occupied, maximize the rental revenue and able to attract customers to visit the mall on regularly basis. For the retailers, they are keen to know the amount, pattern of footfall and profile of customers that visits their store to better optimize their operating and marketing campaigns. For customers, they would like to have better personalized service which will enhance their shopping experience.

WHAT – This focuses on the key problems and current issues. Even with abundance of data or data accessibility, they are not used effectively. There is no visibility of the footfall, traffic density or journey movement of the customers. The limited mall space is not designed to be best used and rental rate are based on qualitative estimates rather than quantitative parameter to differentiate their rental rates. There are poor insights of customers profile, dwell duration and transactions and this stops the mall and retailers from providing better or differentiate and personalized experience for the customers.

WHY – This focuses on the reason on why it matters. Understanding traffic intensity and flow, dwell duration helps understand the patterns of the customers. Rental rates can be based on facts of footfall patterns and trends rather than fixed based on floor or better location. Linking the movement with the profile and transactions of customers allows for customer segmentation with targeted marketing campaigns. HOW – This will identify on what is required to support the key usage scenarios for the key stakeholders. The area density, inter-floor movement will be the first key business objective followed by the journey, duration and profile associated with it. This will support retail management and retailers to better optimize their operations and marketing campaigns and activities. Association rules analysis will further allow link and drill down into transaction behavior, product-mix and market-basket analysis.

PRINCIPLE & METHODOLOGY

T3 principal.png

Key objectives

  • Address challenge of data accessibility
  • Combine and link silos dataset
  • Reproducible application for business user consumption
  • DATA DESCRIPTION

    T3 Datades.PNG



    T3 Dataview.png

    Data Preparation Tasks

  • Data sampling: temporary solution for computational challenge
  • Data cleaning: data types, demographic fields clean up
  • Data joining: merge data from different sources
  • Data transformation: matrix for chord diagram, path for sunburst diagram, transaction table for association analysis, node and edge table for network visualization
  • R Packages for Data Preparation

    tidyverse, dplyr, data.table, lubridate, tidyr, jpeg, reshape2