View source for REO Project Findings Cluster

<br>
<div align="right">
[[ANLY482_AY2017-18_Term_2|<font color="#053B6B" font-family:helvetica><b>Back to ANLY482 AY2017-18 Home Page</b></font>]]
</div>
<!-- Navigation bar -->
{| style="background-color:#FFFFFF; color:##053B6B padding: 5px 0 0 0;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0" |

| style="padding:0.3em; font-family:Arimo; font-size:110%; border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B; text-align:center;" width="10%" |  
[[ANLY482_AY2017-18T2_Group_24|<font face ="Lucida Grande" color="#FFFFFF"><strong> HOME </strong></font>]]
| style="border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B;" width="1%" | &nbsp;

| style="padding:0.3em; font-family:Helvetica; font-size:110%; border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B; text-align:center;" width="10%" |
[[Team_REO|<font  face ="Lucida Grande" color="#FFFFFF"><strong>ABOUT US </strong></font>]]
| style="border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B;" width="1%" | &nbsp;

| style="padding:0.3em; font-family:Arimo; font-size:110%; border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B; text-align:center;" width="10%" | 
[[REO_Project Proposal|<font face ="Lucida Grande" color="#FFFFFF"><strong>PROJECT PROPOSAL</strong></font>]]
| style="border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B;" width="1%" | &nbsp;

| style="padding:0.3em; font-family:Arimo; font-size:110%; border-bottom:5px solid #F9CE2D; border-top:2px solid #FFFFFF; border-left:2px solid #053B6B; background:#FFFFFF; text-align:center;" width="10%" |
[[REO_Project Findings|<font face ="Lucida Grande" color="#053B6B"><strong> PROJECT FINDINGS</strong></font>]]
| style="border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B;" width="1%" | &nbsp;

| style="padding:0.3em; font-family:Arimo; font-size:110%; border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B; text-align:center;" width="10%" |
[[REO_Project_Management|<font  face ="Lucida Grande" color="#FFFFFF"><strong>PROJECT MANAGEMENT </strong></font>]]
| style="border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B;" width="1%" | &nbsp;

| style="padding:0.3em; font-family:Arimo; font-size:110%; border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B; text-align:center;" width="10%" |
[[REO_Documentation|<font  face ="Lucida Grande" color="#FFFFFF"><strong> DOCUMENTATION</strong></font>]]
| style="border-bottom:2px solid #053B6B; border-top:2px solid #053B6B; background:#053B6B;" width="1%" | &nbsp;
|}
<br/>
<!------- Details ---->

{| width="100%" cellspacing="0" cellpadding="0" valign="top" |
| style="padding: 0.25em; font-size: 90%; border-top: 1px solid #cccccc; border-left: 1px solid #cccccc; border-bottom: 1px solid #cccccc; text-align:center; background-color: none; width:30%" | [[REO_Project_Findings| <font color="#053B6B ">Data Preparation</font>]]

| style="padding: 0.25em; font-size: 90%; border-top: 1px solid #cccccc; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-bottom: 1px solid #cccccc; text-align:center; background-color: none; width:30%" | [[REO_Project_Findings_EDA | <font color="#053B6B">Exploratory Data Analysis</font>]]

| style="padding: 0.25em; font-size: 90%; border-top: 1px solid #cccccc; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-bottom: 1px solid #cccccc; text-align:center; background-color: #404040; width:30%" | [[REO_Project_Findings_Cluster | <font color="#ffffff">Cluster Analysis</font>]]
|}

<br>

Having explored the differences between paid and free users, the team decides to perform a cluster analysis via an adapted approach suggested by Punj and Stewart (1983). The approach is:
1.	K-nearest neighbour algorithm to detect outliers
2.	Calculating Cubic Clustering Criterion (CCC) to decide on number of clusters 
3.	K-means to obtain final cluster solution
The team hopes to identify segments within the paid or free users such that REO can formulate strategies for each segment to effectively engage them.
The team decides to exclude entries that have zero enquiries and cobroke requests as enquiries and cobroke requests are the tangible outputs from the platform. Thus, these users have not received any outputs from the platform for the last 6 months which may not be relevant for our cluster analysis.
In addition, the technique of k-means clustering only include entries that are fully filled. As such, the team needs to perform missing value imputation by replacing missing data with zero for the analysis to run.

==Paid Users==
After filtering, there are 4749 users who have paid for the platform and have received at least one enquiry or cobroke request.
===Decision on Clustering Variables===
The team has chosen to use key activity indicators from the previous analysis which include the number of sessions, number of listings posted, number of enquiries and number of Cobroke requests received. The number of sessions and listings are then further broken down into weekday and weekend as well as each individual timeslot.
[[File:REO_Clustering_Variables.png]]

===Examining Clustering Variables===
The team has conducted a descriptive analysis for all clustering variables to identify any outliers and check for skewness.
[[File:REO_paid_Descriptive.png]]
Through identifying extreme outliers that are significantly different from the other points, the team has excluded rows 1328, 3229, 4727 and 6442.

===Transforming the Variables===
Given that the variables are positively skewed, the team has decided to transform the variable through applying cube root. Logarithmic transformation is not used because there are up to 50% of zero entry in “ORGANIC” and “Cobroke_total” variables. However, we noticed that both variables mentioned previously are still slightly skewed because of the number of zeroes.
[[File:REO_paid_Transformation.png]]

===K-Means Clustering===
The cubic clustering criterion (CCC) is used to estimate the number of clusters using Ward’s minimum within-cluster variance method.
[[File:REO_paid_CCC.png]]
Based on the graph above, the number of clusters decided for k is 5 clusters.

===Profiling of Clusters===
[[File:REO_paid_parallel.png]]
The z-score ranking method<sup>4</sup> is a way to profile the clusters. Z-score is calculated to understand how many standard deviation the cluster’s variable mean is away from the variable mean. A positive number refers to being above the variable mean while a negative number refers to being below the variable mean. 
[[File:REO_paid_zscore.png]]
[[File:REO_paid_graph.png]]
[[File:REO_paid_profile.png]]

==Free Users==
After filtering out, there are 5583 users who use the platform for free but receive at least one enquiry or cobroke request.

===Decision on Clustering Variables===
[[File:clustering.png]]

===Examining Clustering Variables===
The team has conducted a descriptive analysis for all clustering variables to identify any outliers and check for skewness.
[[File:REO_free_Descriptive.png]]
The team decides to exclude row 10583 because it is an extreme outlier.

===Transforming the Variables===
Similarly, we performed a cube-root transformation for all clustering variables.
[[File:REO_free_Transformation.png]]

===Screening Outliers===
K-nearest neighbours algorithm is a form of hierarchical clustering to identify any outliers. The team decides to exclude 11 rows with the distance to 1 closest above 4.0.
[[File:REO_free_Outlier.png]]

===K-Means Clustering===
[[File:REO_free_CCC.png]]
Based on the maximum point for cubic clustering criterion, the optimal number of clusters is 7.


===Profiling of Clusters===
[[File:REO_free_parallel.png]]
Similarly, we calculated the z-score for individual variable for each cluster.
[[File:REO_free_zscore.png]]
[[File:REO_free_graph.png]]
[[File:REO_free_profile.png]]