Difference between revisions of "ANLY482 AY2017-18T2 Group14 Interim"
Mq.wang.2014 (talk | contribs) |
Mq.wang.2014 (talk | contribs) |
||
| Line 48: | Line 48: | ||
<!--/Sub Header--> | <!--/Sub Header--> | ||
<br> | <br> | ||
| − | <div style="background: #EAEAEA; padding: 10px; font-weight: bold; text-align:center; line-height: wrap_content; text-indent: 20px;font-size:20px; font-family:helvetica"><font color= #3d3d3d>Data | + | <div style="background: #EAEAEA; padding: 10px; font-weight: bold; text-align:center; line-height: wrap_content; text-indent: 20px;font-size:20px; font-family:helvetica"><font color= #3d3d3d>Understanding Data </font></div> |
| − | |||
| − | |||
| − | |||
| − | |||
<font face ="Open Sans" size=4> | <font face ="Open Sans" size=4> | ||
| Line 66: | Line 62: | ||
</font> | </font> | ||
| + | |||
| + | <br> | ||
| + | <div style="background: #EAEAEA; padding: 10px; font-weight: bold; text-align:center; line-height: wrap_content; text-indent: 20px;font-size:20px; font-family:helvetica"><font color= #3d3d3d>Data Preparation</font></div><br> | ||
| + | <div style="background: #EAEAEA; line-height: 0.3em; border-left: #000000 solid 8px;"><div style="border-left: #FFFFFF solid 5px; padding:15px;"><font face ="Open Sans" color= "black" size="4"><b>Issues Identified</b></font></div></div> | ||
| + | |||
| + | |||
<br> | <br> | ||
Revision as of 20:09, 25 February 2018
In order to understand the structure of OPMS and ODD data sets, we acquired OPMS and ODD fields definition files from our client. However, we found several issues from the definition files provided, which are listed as following:
1. certain variables are not being defined;
2. some variables have different names;
3. all data fields have no indicated data types.
As such, we decided to load sample data of OPMS and ODD into SAS Enterprise Guide to help us understand the metadata.
We do column summary (e.g. one way frequency) for each field and record the data format. Then we consolidate all information to build data definition files for each data set.
1. Unfamiliarity of MSSQL and Power BI:
Prior to this project, we don’t have any prior experience on these tools, Thus, at the beginning of the project, we invested plenty of time in learning and familaring these tools.
2. Lack of domain knowledge:
Domain knowledge is essential in understanding the dataset given, due to the incomplete data definition, we spent a lot of time figuring out the meaning of data, consolidating and documenting the data dictionary.
3. Communicate with users have non-IT background:
We found it is challenging to communicate with users that have limited IT background. Our project sponsor is from operation management. Thus, when we explain some technical complexity to project sponsor, we need to put it into simple and plain words.
4. Data inconsistency (inconsistent data type, data columns, data values):
Data collected from the project sponsor is stored in different places with different formats. Besides, the variable type and variable values are highly inconsistent.
In the next sprint, we will be continuously working on the excel report output0 to output3. As requested by project sponsor, we will keep the origin format for the management team and at the same time, polish origin report to make it more interactive. We aim to finish this by 15 Mar 2018.
Also, we will start working on insight discovery. TBC
