IS428 2017 18T1 Group01 Proposal-version2

PROJECT GROUP

TEAM

PROPOSAL

POSTER

APPLICATION

REPORT

Version 1 | Version 2

INTRODUCTION

StackOverflow is the largest, most trusted online community for developers to learn, share​ ​their programming ​knowledge, and build their careers. This is achieved through questions and answers posted on the site that concerns a wide range of topics related to computer programming. The success of such a platform depends on high quality questions and answers contributed by an engaged community. Thus, some measures to screen for quality includes using ‘badges’ to signify reputation for users, votes on post (upvote or downvote) and using community administrators to close questions that have been answered before or are irrelevant.

MOTIVATION

To sustain its popularity and success, the site needs to continually draw new knowledge from its users in the fast-paced software field. In addition, these said users must constantly be engaged to contribute to the site. As such, StackOverflow can better understand the relationships of its users and the topics that they engage in through data visualisation tools such as network graphs. By understanding social interactions better, it can guide StackOverflow to make necessary changes to keep its user base continually engaged. For instance, if StackOverflow notices a significant group of users with high upvotes for posts but do not post frequently, it can think of how can they encourage such users to use the platform regularly.

OBJECTIVES

Objectives Some of the general problems (questions) that we wish to address are namely:

1. What questions are more likely to be answered and which questions are answered promptly?
2. What are the popular subcommunities and who are the biggest contributors?
   1. What is the distribution of experience levels within each community
   2. Measured by badges
   3. Measured by posts
   4. Measured by upvoted posts (or comments)
   5. Measured by years of experience on StackOverflow
3. Who are the influential members within each StackOverflow subcommunity?

And because we are aspiring data analytics practitioners, here are some specific problems (questions) that we wish to address.

1. What are the little known technologies that are used for data analytics?
2. What do influential members in Stackoverflow know? Are we able to create a competency list of these members?

SELECTED DATASET

Dataset/Source	Data Attributes
Badges https://archive.org/download/stackexchange/stackoverflow.com-Badges.7z	UserId, e.g.: "420" Name, e.g.: "Teacher" Date, e.g.: "2008-09-15T08:55:03.923"
Comments https://archive.org/download/stackexchange/stackoverflow.com-Comments.7z	Id PostId Score Text, e.g.: "@Stu Thompson: Seems possible to me - why not try it?" CreationDate, e.g.:"2008-09-06T08:07:10.730",- UserId
Posts https://archive.org/download/stackexchange/stackoverflow.com-Posts.7z	Id PostTypeId 1: Question 2: Answer,- ParentID (only present if PostTypeId is 2) AcceptedAnswerId (only present if PostTypeId is 1) CreationDate Score ViewCount Body OwnerUserId LastEditorUserId LastEditorDisplayName="Jeff Atwood" LastEditDate="2009-03-05T22:28:34.823" LastActivityDate="2009-03-11T12:51:01.480" CommunityOwnedDate="2009-03-11T12:51:01.480" ClosedDate="2009-03-11T12:51:01.480" Title Tags AnswerCount CommentCount FavoriteCount
Postlinks https://archive.org/download/stackexchange/stackoverflow.com-PostLinks.7z	Id CreationDate PostId RelatedPostId PostLinkTypeId 1: Linked 3: Duplicate
Users https://archive.org/download/stackexchange/stackoverflow.com-Users.7z	Id Reputation CreationDate DisplayName EmailHash LastAccessDate WebsiteUrl Location Age AboutMe Views UpVotes DownVotes
Votes https://archive.org/download/stackexchange/stackoverflow.com-Votes.7z	Id PostId VoteTypeId ` 1`: AcceptedByOriginator ` 2`: UpMod ` 3`: DownMod ` 4`: Offensive ` 5`: Favorite - if VoteTypeId = 5 UserId will be populated ` 6`: Close ` 7`: Reopen ` 8`: BountyStart ` 9`: BountyClose `10`: Deletion `11`: Undeletion `12`: Spam `13`: InformModerator CreationDate UserId (only for VoteTypeId 5) BountyAmount (only for VoteTypeId 9)

BACKGROUND SURVEY OF RELATED WORKS

Related Works	What We Can Learn
Stack Overflow Trends Source: https://insights.stackoverflow.com/trends?utm_source=so-owned&utm_medium=blog&utm_campaign=trends&utm_content=blog-link	See how technologies have trended over time based on use of their tags since 2008 This graph is plotted in a time series and in comparison with other technologies, this allows viewers to quickly grasp which technology is gaining traction and support on the site There is a lack of visibility into changes in user interactions and contributions over time, as technology trends change
Stack Overflow Tag Network Source: https://www.kaggle.com/juliasilge/network-graph/code	The graph is cluttered and static, so it is hard to zoom in to understand each cluster better Overlaps between different clusters are also hard to identify The nodes color are not obviously shown The graph can't compare between the clusters

SKETCHES STORYBOARD

Sketches	How Analyst Can Conduct Analysis
Stack Overflow Trends (Sketches) Stack Overflow Trends (Proposed Layout) Source: http://bl.ocks.org/larskotthoff/4e5dbf8be2c83631a05b	Create a shaded area to outline the different clusters By shading the area is easier to differentiate the different clusters Adding the filter you are able to choose which technologies to compare This graph could answer the questions like "How different technologies related to one another?" and "How different communities of technologies frequently used together?"
Network Analysis (Sketches) Network Analysis (Proposed Layout)	At the bottom of the visualization, it shows a bar graph that represents the interaction between users over a period from the very first interaction to the current date (September 2017) A zoom in function to able to zoom into the nodes and edges Able to filter the different languages community Able to see who is which user answers the most question and which is the most connected in the community

KEY TECHNICAL CHALLENGES

• Storing a extremely large network of StackOverflow posts, comments and users in a graph database and creating indexes for fast retrieval.
• Using StackOverflow API to retrieve information on-demand and transform the data into insight on the visualization.
• Learning how to use the relevant tools for graph analysis such as neo4j, vis.js, sigma.js and D3.js

PROJECT TIMELINE

REFERENCES

http://sotagtrends.com This web application plots the number of questions asked over time by tags on Stackoverflow. Examples of tags include “node.js”, “d3.js” and “python”. Multiple tags can be added by the user and the user can also toggle a relative comparison across the tags. Other interesting features this visualization has includes a moving reference line that slides along the X-axis, showing the exact number of questions asked for each tag.

https://insights.stackoverflow.com/survey/2017 This report created by SO describe insights about SO users using data collected through their annual developer survey. Visualizations used here are simple but effective charts – bar charts, scatter plots and line charts – that describe intriguing phenomena such as preferred programming languages among ethnicities, representation of women in technology roles and correlated technologies used by developers.