IS428 2017 18T1 Group01 Proposal

From Visual Analytics for Business Intelligence
Jump to navigation Jump to search
Awesomeness-Logo.jpg


PROJECT GROUP

 

TEAM

 

PROPOSAL

 

POSTER

 

APPLICATION

 

RESEARCH PAPER

 

Version 1 | Version 2
Stackoverflow.png

INTRODUCTION

StackOverflow is the largest, most trusted online community for developers to learn, share​ ​their programming ​knowledge, and build their careers. This is achieved through questions and answers posted on the site that concerns a wide range of topics related to computer programming. The success of such a platform depends on high quality questions and answers contributed by an engaged community. Thus, some measures to screen for quality includes using ‘badges’ to signify reputation for users, votes on post (upvote or downvote) and using community administrators to close questions that have been answered before or are irrelevant.

MOTIVATION

Our research is motivated by the lack of information on community interaction within Stackoverflow. As Information Sys-tems students who view the site frequently and rely heavily on it for assistance, we re-alised that we are surprisingly unaware of how interaction occurs on the site to keep the community active.

Most visualizations available show the growth and popularity of technologies, but not the underlying users driving these trends. We find it important to study user-to-user interaction as the community plays a very important role in providing useful content to the website. the site to keep the community active.

Apart from the users, the use of multiple tags of technologies (e.g. javascript, jquery, react-native) which are placed on a question posted allow us to study how closely related different technologies are to one another. This is useful in understanding ways to in-tegrate separate technologies with one an-other, and to gauge the popularity of using these technologies together.

OBJECTIVES

Some of the general problems (questions) that we wish to address are namely:

  1. How are the different technologies on Stack Overflow used together?
  2. What are the different clusters of technologies, frequently used together?
  3. Who is likely to answer my question?
  4. Determine which user is more influential in each community?
  5. Do top contributors answer actively and consistently over time?
  6. How does each technology grow over time?
SELECTED DATASET
Dataset/Source Data Attributes
Badges

https://archive.org/download/stackexchange/stackoverflow.com-Badges.7z

  • UserId, e.g.: "420"
  • Name, e.g.: "Teacher"
  • Date, e.g.: "2008-09-15T08:55:03.923"
Comments

https://archive.org/download/stackexchange/stackoverflow.com-Comments.7z

  • Id
  • PostId
  • Score
  • Text, e.g.: "@Stu Thompson: Seems possible to me - why not try it?"
  • CreationDate, e.g.:"2008-09-06T08:07:10.730",- UserId
Posts

https://archive.org/download/stackexchange/stackoverflow.com-Posts.7z

  • Id
  • PostTypeId
  • 1: Question
  • 2: Answer,- ParentID (only present if PostTypeId is 2)
  • AcceptedAnswerId (only present if PostTypeId is 1)
  • CreationDate
  • Score
  • ViewCount
  • Body
  • OwnerUserId
  • LastEditorUserId
  • LastEditorDisplayName="Jeff Atwood"
  • LastEditDate="2009-03-05T22:28:34.823"
  • LastActivityDate="2009-03-11T12:51:01.480"
  • CommunityOwnedDate="2009-03-11T12:51:01.480"
  • ClosedDate="2009-03-11T12:51:01.480"
  • Title
  • Tags
  • AnswerCount
  • CommentCount
  • FavoriteCount
Postlinks

https://archive.org/download/stackexchange/stackoverflow.com-PostLinks.7z

  • Id
  • CreationDate
  • PostId
  • RelatedPostId
  • PostLinkTypeId
  • 1: Linked
  • 3: Duplicate
Users

https://archive.org/download/stackexchange/stackoverflow.com-Users.7z

  • Id
  • Reputation
  • CreationDate
  • DisplayName
  • EmailHash
  • LastAccessDate
  • WebsiteUrl
  • Location
  • Age
  • AboutMe
  • Views
  • UpVotes
  • DownVotes
Votes

https://archive.org/download/stackexchange/stackoverflow.com-Votes.7z

  • Id
  • PostId
  • VoteTypeId
  • ` 1`: AcceptedByOriginator
  • ` 2`: UpMod
  • ` 3`: DownMod
  • ` 4`: Offensive
  • ` 5`: Favorite - if VoteTypeId = 5 UserId will be populated
  • ` 6`: Close
  • ` 7`: Reopen
  • ` 8`: BountyStart
  • ` 9`: BountyClose
  • `10`: Deletion
  • `11`: Undeletion
  • `12`: Spam
  • `13`: InformModerator
  • CreationDate
  • UserId (only for VoteTypeId 5)
  • BountyAmount (only for VoteTypeId 9)
DATA MODEL
Stackoverflow Data Model (2).png

The data model best describes the user and application interactions on the Stack Overflow site. Building the data model allows us to decide what elements should be a node or a relationship.

BACKGROUND SURVEY OF RELATED WORKS
Related Works What We Can Learn
Stack Overflow Trends
Javascript-large-1-1024x621.png

Source: https://insights.stackoverflow.com/trends?utm_source=so-owned&utm_medium=blog&utm_campaign=trends&utm_content=blog-link

  • See how technologies have trended over time based on use of their tags since 2008
  • This graph is plotted in a time series and in comparison with other technologies, this allows viewers to quickly grasp which technology is gaining traction and support on the site
  • There is a lack of visibility into changes in user interactions and contributions over time, as technology trends change
Stack Overflow Tag Network
Image 2017-11-23 19-33-59.png

Source: https://www.kaggle.com/juliasilge/network-graph/code

  • The graph is cluttered and static, so it is hard to zoom in to understand each cluster better
  • Overlaps between different clusters are also hard to identify
  • The nodes color are not obviously shown
  • The graph can't compare between the clusters
SKETCHES STORYBOARD
Sketches How Analyst Can Conduct Analysis
Stack Overflow Trends (Sketches)
Cluster-Tags.jpg

Stack Overflow Trends (Proposed Layout)

ClusterTags.jpg

Source: http://bl.ocks.org/larskotthoff/4e5dbf8be2c83631a05b

  • Create a shaded area to outline the different clusters
  • By shading the area is easier to differentiate the different clusters
  • Adding the filter you are able to choose which technologies to compare
  • This graph could answer the questions like "How different technologies related to one another?" and "How different communities of technologies frequently used together?"
Network Analysis (Sketches)
NetworkTags.jpg

Network Analysis (Proposed Layout)

NetworkCluster.jpg
  • At the bottom of the visualization, it shows a bar graph that represents the interaction between users over a period from the very first interaction to the current date (September 2017)
  • A zoom in function to able to zoom into the nodes and edges
  • Able to filter the different languages community
  • Able to see who is which user answers the most question and which is the most connected in the community
  • Measuring the degree centrality of each user based on the in-degree and out-degree edges, which represent interaction
  • Measuring the betweenness central-ity of each user, which indicates the amount of information flow in the network that will pass through the user
ARCHITECTURE DIAGRAM


Architecture diagram.jpg


KEY TECHNICAL CHALLENGES
  • Storing a extremely large network of StackOverflow posts, comments and users in a graph database and creating indexes for fast retrieval.
  • Using StackOverflow API to retrieve information on-demand and transform the data into insight on the visualization.
  • Learning how to use the relevant tools for graph analysis such as neo4j, sigma.js and D3.js
PROJECT TIMELINE
AWS-Timeline.png
REFERENCES

http://sotagtrends.com This web application plots the number of questions asked over time by tags on Stackoverflow. Examples of tags include “node.js”, “d3.js” and “python”. Multiple tags can be added by the user and the user can also toggle a relative comparison across the tags. Other interesting features this visualization has includes a moving reference line that slides along the X-axis, showing the exact number of questions asked for each tag.

https://insights.stackoverflow.com/survey/2017 This report created by SO describe insights about SO users using data collected through their annual developer survey. Visualizations used here are simple but effective charts – bar charts, scatter plots and line charts – that describe intriguing phenomena such as preferred programming languages among ethnicities, representation of women in technology roles and correlated technologies used by developers.

COMMENTS

Feel free to comments, suggestions and feedbacks to help us improve our project!:D