IS428 2018 19T1 Group11 Proposal

Back to Project Home

HOME

PROPOSAL

POSTER

APPLICATION

REPORT

Version 1 | Version 2

PROBLEM AND MOTIVATION

Despite Grab’s strong presence within the South East Asian rideshare market following the acquisition of Uber in March of this year, the growing number of players within the different fields in which Grab operates in incentivises the company to adopt non-traditional methods to improve its business operations.

While it is important to fulfil the bottom line, a huge determinant of the company’s success stems from the public’s perception and Grab’s positioning in the markets. As such, this project aims to create a systematic method in which Grab can use to understand the public’s sentiment on their product and the company image.

The NLP algorithm used by Grab to describe the public’s sentiments follows the Latent Dirichlet Allocation (LDA) model, a generative statistical model that allows sets of observations to be explained by unobserved groups, or topics, that explain how some parts of the data are similar. With this model, Grab hopes to identify latent topics of interest and understand the public’s perception of the topic. Grab can therefore make use of this information to address any inadequacies in their business practices, create more effective marketing campaigns and improve their business operations to build a stronger overall brand image.

OBJECTIVES

The objective of the visualization is to bridge the gap between the analytics and business teams. While the findings from the LDA model might be intuitive for those from the analytics team, the business users may find it difficult to internalize these findings. Hence, we aim to create a scalable way to present these findings to the business users in an easy to understand format.

As mentioned before, we hope that by employing the LDA model, the business teams will be able to get a sense of the current perception of Grab’s products by the community. In the model, the user chooses an input of the total number of topics that he or she believes is an ideal balance between granularity and actionability (i.e too many topics may result in low actionability, while too little topics may result in not enough cause for action). As the ideal number of topics are often subjective, our proposed visualization will have the option for users to adjust the number of topics based on what he/she believes is ideal. We will also be including a covariance score plot that can guide users towards the ideal number of topics.

DATA SOURCE

Data Source
Data used is obtained from web scraping of various social media platforms such as Instagram, Twitter, Reddit and Google Play.
The data set consists of 9000 comments that were scraped and collected.

Data Attributes
The following is a snapshot of the data collected, and a description of the data attributes:

Figure 1: Comments Dataset

Data Attributes	Description of attributes
Document	Comments scraped may consist of more than a sentence each. They are separated and identified by documents. Hence, a document represents a sentence of comment.
Dominant_Topic	Dominant topic refers to the topic that the document will most likely be sorted into.
Topic_Perc_Contrib	The probability that the comment will be found in the topic amongst all other comments with similar keywords.
Keywords	Keywords that belong in each of the topics.
Text	Words in each comment after the removal of stop words (eg. the, is, to, on etc).
Original_Comment	Original comment.
Comment_Date	Date that the comment was posted.

BACKGROUND SURVEY OF RELATED WORKS

Related Works	What We Can Learn
Tweet Sentiment Visualisation Source: https://www.csc2.ncsu.edu/faculty/healey/tweet_viz/tweet_app/	Using a sentiment dictionary to estimate the sentiment of each tweet Scaling sentiments from negative to positive, where blue is unpleasant and green is pleasant Shades of tweets are dependent on whether the tweet is an active tweet or sedate tweet
Tweet topic cluster visualisation Source: https://www.csc2.ncsu.edu/faculty/healey/tweet_viz/tweet_app/	Common words are grouped into topic clusters Keywords in a cluster indicates the topic At the same time, words that are not categorised in a topic are separated
Tweet sentiment across time Source: https://www.csc2.ncsu.edu/faculty/healey/tweet_viz/tweet_app/	Shows sentiment of words over time using a time series line graph Can be used to observe the change in topic interest over time Idea can be adapted to comments in every cluster of comments

STORYBOARD

Sketches	Description of Approach
Visualisation 1: User modelling interface	Idea: Users are able to switch between the number of clusters they wish to output using coherence value as a guide. Coherence score chart Cluster count (for user input) Relative size of clusters to each other Top 5 words per cluster Suggested cluster topic based on keyword that appears most frequently
Visualisation 2: Topic Cluster Visualisation Option 1 Inspired by: https://www.csc2.ncsu.edu/faculty/healey/tweet_viz/	Idea: Show how topics change in discussion frequency and sentiment over time. Also observe birth and death of topics. Grab service filter Sentiment legend* Topic frequency legend Date scroll bar Selected Topic Actual comments (comments highlighted based on selected topic) Details of selected topic *subject to data availability
Visualisation 2: Topic Cluster Visualisation Option 2	Idea: Increased granularity compared to Option 1, to show distribution of individual comment sentiments within each topic cluster. Same features as above
Visualisation 2: Topic Cluster Visualisation Option 3 Inspired by: http://lcs.ios.ac.cn/~shil/paper/VISA_VINCI.pdf	Idea: Allow snapshot view of changes in topic frequency and sentiment across the whole time period. Grab service filter Date scroll bar Sentiment legend* Selected Topic Actual comments (comments highlighted based on selected topic Details of selected topic *subject to data availability

KEY TECHNICAL CHALLENGES

Key challenges	Proposed solution to overcome challenges
Data cleaning and ensuring good data quality	Take time to understand and explore sponsored data Understand procedures taken by sponsor to process data before sending it to us Validate quality of data before proceeding
Lack of experience using relevant tools for analysis such as RShiny	Attend the R workshop conducted by Prof. Kam Independent learning through online tutorials such as DataCamp Seek advice from seniors and ask on online forums
Determining the most effective way to visualise the data	Brainstorm and design storyboard before implementing anything Consult sponsor on requirements Consult Prof. Kam on visualisation direction Research on similar visualizations to determine what the best approach is

PROJECT TIMELINE

TECHNOLOGIES AND TOOLS

Technology and Tools	Explanation
	We will primarily be using Shiny for our visualization. Shiny is an open source R package that provides an elegant and powerful web framework for building web applications using R
	We will be building the machine learning model and visualization using R Studio
	We will be using Photoshop to design our poster

REFERENCES

https://www.csc2.ncsu.edu/faculty/healey/tweet_viz/
https://rstudio-pubs-static.s3.amazonaws.com/236186_d311ea00291d42509864aa0a77d340e8.html
http://lcs.ios.ac.cn/~shil/paper/VISA_VINCI.pdf
These pages have inspired numerous alternatives to visualising topic sentiments

COMMENTS

Feel free to comment and leave suggestions and feedback to help us improve our project!:D