Difference between revisions of "Group 8 Overview"
| Line 25: | Line 25: | ||
=== Time series Explorer === | === Time series Explorer === | ||
| − | Time | + | Time series data has always played an important role in understanding and evaluating past behaviours. The usage of time-series information has allowed companies and organizations to tune their operations instead of simply improving by trial and error. Time series data have also been used to evaluate whether an organization has grown throughout the years of operation. The analysis techniques for time-bound data help to describe and explore insights, while predicting the future outcomes such as revenues and profits. |
| − | One of the important example measures that can allow us to get a glimpse would be the Consumer Price Index (CPI). | + | This category of data analysis is not only constrained to the business world. A country's economic data is also a rich pool of information that can be analysed for insights. One of the important example measures that can allow us to get a glimpse would be the Consumer Price Index (CPI). Using this easily available measure, we can explore living standards of a country's citizens by understanding the factors affecting their day-to-day lives. |
=== Use Case === | === Use Case === | ||
| − | The Consumer Price Index (CPI), is a critical indicator to assess | + | The Consumer Price Index (CPI), is a critical indicator to assess a country's consumer price inflation. To profile the weighted average price changes for households' cost of living, Singapore's CPI adopts a fixed basket of residents' commonly consumption goods and services. About 6,600 brands/varieties from 4,200 outlets are selected in the 2014-based CPI (the dataset for our study). At basket level, the composition of goods and services can be categorized into 10 major divisions, which are listed below: |
Food, Clothing & Footwear, Housing & Utilities, Household Durables And Services, Health Care, | Food, Clothing & Footwear, Housing & Utilities, Household Durables And Services, Health Care, | ||
| Line 37: | Line 37: | ||
[[Image:SG_2014_BASED_CPI.png|500px|right|width="100%"]] | [[Image:SG_2014_BASED_CPI.png|500px|right|width="100%"]] | ||
| − | To construct CPI, two main types of data are required - the price data of | + | To construct the CPI, two main types of data are required - the sample price data of the good or service, and the weighting data to represent the proportion of different categories' expenditure. The price data is gathered through a combination of data collection modes, while the frequency of price collection depends on the price behavior of the item. The weighting data is derived from the expenditure values collected in the Household Expenditure Survey, the latest one came from HES 2012/13, updated to 2014 values by taking into account price changes between 2012/13 and 2014. |
As a Price index, CPI can also be affected by other types of costs in Singapore. Indicators like COE Bidding Price, Import and Export price index, Exchange Rates, are assumed to have interesting relationships with CPI which are worthy of further study. These data sources are also acceptable within the proposed system design within certain format restrictions. | As a Price index, CPI can also be affected by other types of costs in Singapore. Indicators like COE Bidding Price, Import and Export price index, Exchange Rates, are assumed to have interesting relationships with CPI which are worthy of further study. These data sources are also acceptable within the proposed system design within certain format restrictions. | ||
| Line 43: | Line 43: | ||
== Motivation == | == Motivation == | ||
| − | + | During our personal data analysis research and experiences, we discovered a lack of freely available analysis tools that can help us optimize the parameter settings of time-series models. The result is a large amount of time and effort utilized to enter in different combinations of parameters and waiting for the models to be trained. The time series system that we have in mind would need to help us estimate the model's accuracy rates automatically while we perform our data analyses, so we can simply choose the best models for comparison. | |
| − | We would | + | Our team would also like to create exploratory and predictive models, which showcase these complex time-related trends of Singapore's CPI throughout the years (1990-2017) for different categories. We would like to use time-series visualization techniques such as tables and line charts representing Trend, Seasonality and Random to investigate any insights and to display potential forecasts to the audience. |
| + | |||
| + | We will apply system design principles to make the proposed system accept any form of generic time-series data. This is to allow flexibility of the system and expands its scope of usage. | ||
There are several reasons why we found this project interesting: | There are several reasons why we found this project interesting: | ||
| Line 51: | Line 53: | ||
* General exploration of time-series data. The project allows us to learn and re-learn time-series analysis techniques and concepts. This gives us the opportunity to let us put our theoretical knowledge into practical use. | * General exploration of time-series data. The project allows us to learn and re-learn time-series analysis techniques and concepts. This gives us the opportunity to let us put our theoretical knowledge into practical use. | ||
* Closer to our daily standards of living, we wanted to understand what are the current categories that make up a country's CPI index and subsequently its impact on its citizens. | * Closer to our daily standards of living, we wanted to understand what are the current categories that make up a country's CPI index and subsequently its impact on its citizens. | ||
| − | * We wanted to explore whether different periods of time would indicate the different price index values in a cyclical manner. This would also allow us to understand the overall direction prices are taking for every goods & services category. | + | * We also wanted to explore whether different periods of time would indicate the different price index values in a cyclical manner. This would also allow us to understand the overall direction prices are taking, for every goods & services category in Singapore. |
| − | Even though Singapore is a relatively young country, it is able to provide rich data to help us explore these interesting observations. This is also in part due to the government initiative of | + | Even though Singapore is a relatively young country, it is able to provide rich data to help us explore these interesting observations. This is also in part due to the government initiative of 'SmartNation.sg'. |
== Objectives == | == Objectives == | ||
| Line 60: | Line 62: | ||
2. Discover data insights using visualization and interactivity that cannot be easily represented using raw data. | 2. Discover data insights using visualization and interactivity that cannot be easily represented using raw data. | ||
| − | 3. Make use of freely available Singapore economic data to arouse the interests of potential viewers and increase their curiosity on | + | 3. Make use of freely available Singapore economic data to arouse the interests of potential viewers and increase their curiosity on the current state of Singapore's consumer goods and services. |
== Data Source == | == Data Source == | ||
| Line 80: | Line 82: | ||
== Application == | == Application == | ||
| + | The proposed system would have three major functions: | ||
| + | i) Data Manipulation: | ||
| + | |||
| + | System users would be able to upload time-series data files within certain formats. The interface provided would allow some forms of data transformations such as the transposition of columns, the generation of an index column as a substitute for a missing datetime column, and the indication of missing time series periods. These functionalities are required because time series analysis needs data to be indexed by a form of datetime field. Metadata information of the uploaded dataset would be displayed for easy viewing and the uploaded dataset can also be previewed from the main panel. | ||
| + | |||
| + | ii) Data Exploration: | ||
| + | |||
| + | The system should allow the functionality to decompose time-series information into its constituent parts: Observation, Seasonal, Trend, Random (Noise). From the separate parts, users can understand the different time-series patterns and derive insights. The system will also provide several filter functions such as denoting whether the data is additive or multiplicative trend, start and end dates of the data, and the frequency of the time-series periods. | ||
| + | |||
| + | iii) Forecasting: | ||
| + | |||
| + | The system should also be able to forecast time-series data that have been filtered out from the Data Exploration feature. The forecasting techniques will utilize Exponential Smoothing and ARIMA techniques to perform predictions. An optimization algorithm will be used along with existing packages to find the best set of parameters and the top 3 models of each technique will be selected based on their AIC, BIC values. Once selected, the models can then be graphed on the page as a comparison. | ||
== References == | == References == | ||
1. https://data.gov.sg/dataset/consumer-price-index-monthly?view_id=0063aa5a-c5de-4c74-94be-b9ec443878be&resource_id=67d08d6b-2efa-4825-8bdb-667d23b7285e | 1. https://data.gov.sg/dataset/consumer-price-index-monthly?view_id=0063aa5a-c5de-4c74-94be-b9ec443878be&resource_id=67d08d6b-2efa-4825-8bdb-667d23b7285e | ||
| − | |||
2. https://secure.mas.gov.sg/msb/ExchangeRates.aspx | 2. https://secure.mas.gov.sg/msb/ExchangeRates.aspx | ||
| − | |||
3. https://insights-ceicdata-com.libproxy.smu.edu.sg/Untitled-insight/views | 3. https://insights-ceicdata-com.libproxy.smu.edu.sg/Untitled-insight/views | ||
| − | |||
4. http://tralvex.com/pub/cars/coe.htm | 4. http://tralvex.com/pub/cars/coe.htm | ||
Revision as of 15:55, 28 November 2017
|
|
|
|
|
Contents
Background
Time series Explorer
Time series data has always played an important role in understanding and evaluating past behaviours. The usage of time-series information has allowed companies and organizations to tune their operations instead of simply improving by trial and error. Time series data have also been used to evaluate whether an organization has grown throughout the years of operation. The analysis techniques for time-bound data help to describe and explore insights, while predicting the future outcomes such as revenues and profits.
This category of data analysis is not only constrained to the business world. A country's economic data is also a rich pool of information that can be analysed for insights. One of the important example measures that can allow us to get a glimpse would be the Consumer Price Index (CPI). Using this easily available measure, we can explore living standards of a country's citizens by understanding the factors affecting their day-to-day lives.
Use Case
The Consumer Price Index (CPI), is a critical indicator to assess a country's consumer price inflation. To profile the weighted average price changes for households' cost of living, Singapore's CPI adopts a fixed basket of residents' commonly consumption goods and services. About 6,600 brands/varieties from 4,200 outlets are selected in the 2014-based CPI (the dataset for our study). At basket level, the composition of goods and services can be categorized into 10 major divisions, which are listed below:
Food, Clothing & Footwear, Housing & Utilities, Household Durables And Services, Health Care,
Transport, Communication, Recreation & Culture, Education, Miscellaneous Goods & Services
To construct the CPI, two main types of data are required - the sample price data of the good or service, and the weighting data to represent the proportion of different categories' expenditure. The price data is gathered through a combination of data collection modes, while the frequency of price collection depends on the price behavior of the item. The weighting data is derived from the expenditure values collected in the Household Expenditure Survey, the latest one came from HES 2012/13, updated to 2014 values by taking into account price changes between 2012/13 and 2014.
As a Price index, CPI can also be affected by other types of costs in Singapore. Indicators like COE Bidding Price, Import and Export price index, Exchange Rates, are assumed to have interesting relationships with CPI which are worthy of further study. These data sources are also acceptable within the proposed system design within certain format restrictions.
Motivation
During our personal data analysis research and experiences, we discovered a lack of freely available analysis tools that can help us optimize the parameter settings of time-series models. The result is a large amount of time and effort utilized to enter in different combinations of parameters and waiting for the models to be trained. The time series system that we have in mind would need to help us estimate the model's accuracy rates automatically while we perform our data analyses, so we can simply choose the best models for comparison.
Our team would also like to create exploratory and predictive models, which showcase these complex time-related trends of Singapore's CPI throughout the years (1990-2017) for different categories. We would like to use time-series visualization techniques such as tables and line charts representing Trend, Seasonality and Random to investigate any insights and to display potential forecasts to the audience.
We will apply system design principles to make the proposed system accept any form of generic time-series data. This is to allow flexibility of the system and expands its scope of usage.
There are several reasons why we found this project interesting:
- General exploration of time-series data. The project allows us to learn and re-learn time-series analysis techniques and concepts. This gives us the opportunity to let us put our theoretical knowledge into practical use.
- Closer to our daily standards of living, we wanted to understand what are the current categories that make up a country's CPI index and subsequently its impact on its citizens.
- We also wanted to explore whether different periods of time would indicate the different price index values in a cyclical manner. This would also allow us to understand the overall direction prices are taking, for every goods & services category in Singapore.
Even though Singapore is a relatively young country, it is able to provide rich data to help us explore these interesting observations. This is also in part due to the government initiative of 'SmartNation.sg'.
Objectives
1. Provide interactive platform to illustrate the trends and seasonalities within given time-series data (i.e. Singapore's CPI).
2. Discover data insights using visualization and interactivity that cannot be easily represented using raw data.
3. Make use of freely available Singapore economic data to arouse the interests of potential viewers and increase their curiosity on the current state of Singapore's consumer goods and services.
Data Source
The Consumer Price Index (CPI) data is extracted from data.gov.sg in a monthly format which reveals the figures from January 1961 to August 2017, while the index reference period is 2014. The data has an overall index representing changes in the price level of the whole basket with all items considered, and can also be drilled down to sub-indices and sub-sub-indices for different categories and sub-categories of goods and services. For our system analysis, we plan to use filtered data from 1990 onwards.
Analysis Methods
Exploratory Analysis
We will explore the different trends of time-series data provided by the various economic data sets (Period cyclicity and seasonality). Different interactions of identified attributes might provide certain data insights that we can use for our analysis.
Explanatory Analysis
Relationships between our data will be explained based on our understanding of possible real-world events or causes. For example, we might be able to explain the difference in CPI between the months of June and December due to the holiday seasons causing an increase of demand for clothing in December.
Predictive Analysis
Due to the repeating nature of cyclic and seasonal time-series data, we can use the identified trends to predict the various CPI index values for future dates. Potential analytics techniques include different models of exponential smoothing and ARIMA.
Application
The proposed system would have three major functions: i) Data Manipulation:
System users would be able to upload time-series data files within certain formats. The interface provided would allow some forms of data transformations such as the transposition of columns, the generation of an index column as a substitute for a missing datetime column, and the indication of missing time series periods. These functionalities are required because time series analysis needs data to be indexed by a form of datetime field. Metadata information of the uploaded dataset would be displayed for easy viewing and the uploaded dataset can also be previewed from the main panel.
ii) Data Exploration:
The system should allow the functionality to decompose time-series information into its constituent parts: Observation, Seasonal, Trend, Random (Noise). From the separate parts, users can understand the different time-series patterns and derive insights. The system will also provide several filter functions such as denoting whether the data is additive or multiplicative trend, start and end dates of the data, and the frequency of the time-series periods.
iii) Forecasting:
The system should also be able to forecast time-series data that have been filtered out from the Data Exploration feature. The forecasting techniques will utilize Exponential Smoothing and ARIMA techniques to perform predictions. An optimization algorithm will be used along with existing packages to find the best set of parameters and the top 3 models of each technique will be selected based on their AIC, BIC values. Once selected, the models can then be graphed on the page as a comparison.
References
1. https://data.gov.sg/dataset/consumer-price-index-monthly?view_id=0063aa5a-c5de-4c74-94be-b9ec443878be&resource_id=67d08d6b-2efa-4825-8bdb-667d23b7285e 2. https://secure.mas.gov.sg/msb/ExchangeRates.aspx 3. https://insights-ceicdata-com.libproxy.smu.edu.sg/Untitled-insight/views 4. http://tralvex.com/pub/cars/coe.htm
