Difference between revisions of "ANLY482 AY2017-18T2 Group14 Methodology"
Mq.wang.2014 (talk | contribs) |
Mq.wang.2014 (talk | contribs) |
||
| Line 50: | Line 50: | ||
<br/> | <br/> | ||
<div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Database Management</b></font></div></div> | <div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Database Management</b></font></div></div> | ||
| + | <font face ="Open Sans" size=4> | ||
Currently, the data are stored separately in two different places, one is stored in the mainframe whereas one is stored in newly added system which is still in negotiating process. The separation of the data has no doubts increase the difficulty of database management. The main data we going to use in our project, on-demand delivery data (aka., ODD data) is retrieved from new system. The major tools used for these two data are Microsoft Access and Microsoft Excel. | Currently, the data are stored separately in two different places, one is stored in the mainframe whereas one is stored in newly added system which is still in negotiating process. The separation of the data has no doubts increase the difficulty of database management. The main data we going to use in our project, on-demand delivery data (aka., ODD data) is retrieved from new system. The major tools used for these two data are Microsoft Access and Microsoft Excel. | ||
| Line 55: | Line 56: | ||
Noticed that there are differences exist in two set of data, we will firstly extract the raw data from Microsoft Access then perform any needed data cleaning. Next, we will create different tables for different data, and set up proper relations among tables. By doing this, we can provide our client flexibilities to maintain (i.e. CRUD) their operating data. Lastly, imported the cleaned data into the already set up database system. | Noticed that there are differences exist in two set of data, we will firstly extract the raw data from Microsoft Access then perform any needed data cleaning. Next, we will create different tables for different data, and set up proper relations among tables. By doing this, we can provide our client flexibilities to maintain (i.e. CRUD) their operating data. Lastly, imported the cleaned data into the already set up database system. | ||
| − | + | </font> | |
<br/> | <br/> | ||
<div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Exploratory Analysis and Model Building</b></font></div></div> | <div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Exploratory Analysis and Model Building</b></font></div></div> | ||
| + | <font face ="Open Sans" size=4> | ||
After setting up the database, we will begin analysing the data by performing exploratory data analysis using Power BI/Excel. Then, we will use some analytical models, such as geospatial analysis, clustering, regression, in order to help us reveal any hidden patterns not known by the client. | After setting up the database, we will begin analysing the data by performing exploratory data analysis using Power BI/Excel. Then, we will use some analytical models, such as geospatial analysis, clustering, regression, in order to help us reveal any hidden patterns not known by the client. | ||
| + | </font> | ||
<br/> | <br/> | ||
<div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Data Visualization and Reporting</b></font></div></div> | <div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Data Visualization and Reporting</b></font></div></div> | ||
| + | <font face ="Open Sans" size=4> | ||
Microsoft Excel is currently used for routine data analysis and reporting but it only works well when the data is small. However, with the rapidly-growing data set used by the client, Microsoft Excel becomes a constraint that limits the data processing capacity and their work efficiency. Hence, in this project, Power BI, an interactive data visualization BI tools will be used to create an automatic dashboard for data analysis and reporting. | Microsoft Excel is currently used for routine data analysis and reporting but it only works well when the data is small. However, with the rapidly-growing data set used by the client, Microsoft Excel becomes a constraint that limits the data processing capacity and their work efficiency. Hence, in this project, Power BI, an interactive data visualization BI tools will be used to create an automatic dashboard for data analysis and reporting. | ||
| − | + | </font> | |
<br/> | <br/> | ||
<div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Technology and Tools </b></font></div></div> | <div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Technology and Tools </b></font></div></div> | ||
| − | + | <font face ="Open Sans" size=4> | |
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
| Line 84: | Line 88: | ||
| JMP|| JMP is a data analysis software that is used for preliminary data analysis and data cleaning. | | JMP|| JMP is a data analysis software that is used for preliminary data analysis and data cleaning. | ||
|} | |} | ||
| + | </font> | ||
Latest revision as of 16:20, 12 April 2018
Currently, the data are stored separately in two different places, one is stored in the mainframe whereas one is stored in newly added system which is still in negotiating process. The separation of the data has no doubts increase the difficulty of database management. The main data we going to use in our project, on-demand delivery data (aka., ODD data) is retrieved from new system. The major tools used for these two data are Microsoft Access and Microsoft Excel.
In our project, we will assist the team to build an integrated database where they can store two set of data together in unified format. To achieve this, we will use the open-source relational database management system- MSSQL for database management.
Noticed that there are differences exist in two set of data, we will firstly extract the raw data from Microsoft Access then perform any needed data cleaning. Next, we will create different tables for different data, and set up proper relations among tables. By doing this, we can provide our client flexibilities to maintain (i.e. CRUD) their operating data. Lastly, imported the cleaned data into the already set up database system.
After setting up the database, we will begin analysing the data by performing exploratory data analysis using Power BI/Excel. Then, we will use some analytical models, such as geospatial analysis, clustering, regression, in order to help us reveal any hidden patterns not known by the client.
Microsoft Excel is currently used for routine data analysis and reporting but it only works well when the data is small. However, with the rapidly-growing data set used by the client, Microsoft Excel becomes a constraint that limits the data processing capacity and their work efficiency. Hence, in this project, Power BI, an interactive data visualization BI tools will be used to create an automatic dashboard for data analysis and reporting.
| Technology & Tools | Description |
| PowerBI | PowerBI is a suite of business analytics tools that will be used to generate excel reports and visualizations for ODD report. |
| Microsoft SQL Server | Microsoft SQL Server is relational database management system that will used to manage data collected from the project sponsor. |
| Tableau | Tableau is a Business Intelligence and Analytics tool that will be used for exploratory data analysis and dashboard development. |
| SAS Enterprise Guide | In metadata construction, SAS Enterprise Guide is used to summarize each data column. |
| JMP | JMP is a data analysis software that is used for preliminary data analysis and data cleaning. |
