ANLY482 AY2017-18T2 Group30 LDA-Blog Post

From Analytics Practicum
Revision as of 15:44, 7 February 2018 by Eric.yeo.2014 (talk | contribs) (Created page with "<!--Team Logo--> center|300px| <!--End of Team Logo--> <br/> <!--Main Navigation--> <center> {|style="background-color:#5A6B96; color:#5A6B96; width=...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
APex Logo.PNG


HOME ABOUT US PROJECT OVERVIEW PROJECT FINDINGS PROJECT MANAGEMENT DOCUMENTATION MAIN PAGE
Facebook Post Facebook Video Youtube Instagram Blog Post


Data Source

Facebook
For data files from Facebook Insights Data Export (Post Level), the sponsor provided exported data from different periods of the year, with different metric tabs in Excel format. The tabs included are:

  1. Key Metrics
  2. Lifetime: Number of unique people who have created a story about your Page post by interacting with it (unique users)
  3. Lifetime: Number of people who have clicked anywhere in your post, by type (unique users)
  4. Lifetime: Number of people who have given negative feedback on your post, by type (unique users)

For data files from Facebook Insights Data Export (Video Post), the sponsor provided exported data from different periods of the year, with different metric tabs in Excel format. The tabs included are:

  1. Lifetime Post Total Impression/Reach/Views
  2. Geographic Views
  3. Demographic Views
  4. Lifetime Post Toal Views by (page_owned / Shared)

YouTube
For data files from YouTube(Watch Time), the sponsor provided exported data for Watch Time, with different metric tabs in Excel format. The tabs included are:

  1. Video
  2. Geography
  3. Date
  4. Subscription Status
  5. Youtube Product
  6. Device Type
  7. Subtitles and CC
  8. Video Information Language


For data files from YouTube(Demographics), the sponsor provided exported data for watch time for different Demographic, with different metric tabs in Excel format. The tabs included are:

  1. Viewer Age
  2. Viewer Gender


For data files from YouTube(Traffic Sources), the sponsor provided exported data for watch time from different traffic source type


Instagram
To retrieve data from the company's instagram, we made use of a web-scraping script from Github. We made modifications to the script to include timestamp as well as caption, the data includes:

  • Caption
  • Timestamp
  • Img URL
  • Tags
  • No. of Likes
  • No. of Comments


Blog
To retrieve data from the company's posts, we used Scrapy, a fast and powerful open-sourced web-scraper to extract data from the blog. We collected data from the beginning of the first blog post, with the following information:

  • Timestamp
  • Author(s)
  • Headline
  • Category
  • URL
  • Tags


Data Preparation

To help us have an overview of the data throughout the year, we consolidated the various tabs, whilst concatenating the various periods of data for the same columns, into one combined file. This was carried out using the software, IBM JMP Pro, in the following steps:

  • With Post ID, Permalink (permanent link of the campaign content), Post Message, Type, Countries and Posted columns as key identifiers among the different tabs for the excel files, we appended desired columns from the other tabs to the end of the Key Metrics. They included the Share, Like, Comment columns from Tab 2; Other Clicks, Link Clicks, Photo View, Video Play columns from Tab 3; Hide_Clicks , Hide_all_clicks, Unlike_page_clicks, report_spam_clicks columns from Tab 4.
    This was conducted using the Tables > Join function, with “Matching Specification” as the key identifiers and “Output Columns” of the appended desired columns.
  • Next, for each period of data files (appended with new columns) from multiple tabs, we concatenate the data across different time periods to have a full year collection of data.
    This was conducted using the Tables > Concatenate function, while adding multiple data tables into “Data Tables to be Concatenated”.
  • Finally, we check for missing data in the different columns. For example, under the column Type, we have five different types, namely: Link, Photo, Shared Video, Status and Video. However, in the instances of missing data, we will cross check with the permalink of the campaign post, and check the Type of medium was posted and fill it in accordingly.


Data Cleaning

Instagram Data
After scraping the data, we realised that the data needed cleaning. The indexes of the column values were off as seen here: (image) We also concatenated the "tags" into a single column.


Exploratory Data Analysis



Final Application: Learning Dashboard