Difference between revisions of "GeViz"

From Visual Analytics for Business Intelligence
Jump to navigation Jump to search
Line 70: Line 70:
 
===Text Classification using Support Vector Classifier (SVC) ===
 
===Text Classification using Support Vector Classifier (SVC) ===
 
One of the key challenges of working with the provided procurement dataset is the absence of categorization of each procurement transaction. <br/>
 
One of the key challenges of working with the provided procurement dataset is the absence of categorization of each procurement transaction. <br/>
Instead of labelling manually, we applied <b>machine learning</b> to classify the tender descriptions into different categories. We firstly scraped the procurement descriptions and categories from GeBiz website using <b> Selenium</b> and <b> BeautifulSoup libraries in Python to be used as the training and validation dataset in our Support Vector Classifier model. We were able to achieve 90% for training accuracy before performing the categorization prediction.
+
Instead of labelling manually, we applied <b>machine learning</b> to classify the tender descriptions into different categories. We firstly scraped the procurement descriptions and categories from GeBiz website using <b> Selenium</b> and <b> BeautifulSoup </b> libraries in Python to be used as the training and validation dataset in our Support Vector Classifier model. We were able to achieve <b> 90% for training accuracy </b> before performing the categorization prediction.
  
 
==<div style="background:#143c67; padding:15px; font-weight: bold; line-height: 0.3em;letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Lato"><center>BACKGROUND SURVEY</center></font></div>==
 
==<div style="background:#143c67; padding:15px; font-weight: bold; line-height: 0.3em;letter-spacing:0.5em;font-size:20px"><font color=#fbfcfd face="Lato"><center>BACKGROUND SURVEY</center></font></div>==

Revision as of 22:43, 24 November 2018

Geviz.png


Team

 

Proposal

 

Poster

 

Application

 

Research Paper


PROBLEM & MOTIVATION


GeBIZ is a Singapore Government’s one-stop e-procurement portal which facilitates the entire procurement lifecycle and revenue tender activities between Singapore government and local and overseas supplies electronically since June 2000. The purpose of the portal is to be transparency, encourage fair and open competition and to generate demands / quotations. With this portal, more companies are tendering for projects hence increasing the competitiveness.

We aim to create a visualisation which benefits businesses in crafting bidding prices and increasing their chances in winning the government contracts and tender. In addition, we hope to give an overview and identify interesting patterns of government spending.

OBJECTIVES


In this project, we are creating a visualisation that is able to show the following:

  • Government investment analysis
  • The tender distribution of projects to different type of business
  • Identify the price range when bidding for projects


SELECTED DATASETS

The following datasets will be used for analysis , as elaborated below:

Dataset/Source Data Attributes Rationale of Usage
Government Procurement Data (https://data.gov.sg/dataset/government-procurement)
  • Tender No
  • Agency
  • Tender Description
  • Award Date
  • Tender Detail Status
  • Supplier Name
  • Awarded Amount
To gain information on government procurement such as tender description, amount and supplier information
Ministry and Agencies List
  • Ministry
  • Agency
We will be looking through the Singapore Government Directory (https://www.gov.sg/sgdi/ministries) to categorise the agencies into their respective ministries. This will allow us to visualise the procurement spending on a ministry level.


Text Classification using Support Vector Classifier (SVC)

One of the key challenges of working with the provided procurement dataset is the absence of categorization of each procurement transaction.
Instead of labelling manually, we applied machine learning to classify the tender descriptions into different categories. We firstly scraped the procurement descriptions and categories from GeBiz website using Selenium and BeautifulSoup libraries in Python to be used as the training and validation dataset in our Support Vector Classifier model. We were able to achieve 90% for training accuracy before performing the categorization prediction.

BACKGROUND SURVEY


Some of these visualizations that we draw inspiration from, are as follows: To be filled!

CONSIDERATION & VISUAL SELECTION


To be filled!

BRAINSTORMING SESSIONS



TECHNOLOGIES

Tools and technologies

Tools used.png


Data Architecture

Data architecture.png


KEY CHALLENGES

The following are some of the key technical challenges that we may face throughout the course of the project:

Key Challenges Mitigation Plan
Unfamiliarity with Javascript Rshiny libraries )
  • Attend R Shiny Workshop
  • Independent learning via online resources
  • Ask team mates for help
Unfamiliarity with R Libraries for Machine Learning and Selenium
  • Independent learning via online resources
  • Clean, transform and analyse data together
Data Cleaning and Transformation
  • Need to crawl data on website to obtain company information
  • Clean, transform the data together


TIMELINE


To be filled!



COMMENTS


Feel free to leave us some comments so that we can improve! We dont bite :)

No. Name Date Comments
1. Insert your name here Insert date here Insert comment here
2. Insert your name here Insert date here Insert comment here
3. Insert your name here Insert date here Insert comment here