IS428 2016-17 Term1 Assign3 Tan Kee Hock
To be a Visual Detective: Detecting spatio-temporal patterns
Contents
Overview
After the successful resolution of the 2014 kidnapping at GAStech’s Abila, Kronos office, GAStech officials determined that Abila offices needed a significant upgrade. At the end of 2015, the growing company moved into a new, state-of-the-art three-story building near their previous location. Even though the employee morale rose somewhat with the excitement of the new building, there are still a few disgruntled employees in the company.
The new office is built to the highest energy efficiency standard, but as with any new building, there are still several HVAC issues to work out. The building is divided into several HVAC (heating, ventilation, and air conditioning) zones. Each zone is instrumented with sensors that report building temperatures, heating and cooling system status values, and concentration levels of various chemicals such as carbon dioxide (abbreviated CO2) and hazium (abbreviated Haz), a recently discovered and possibly dangerous chemical. CEO Sten Sanjorge Jr. has read about hazium and requested that these sensors be included. However, they are very new and very expensive, so GAStech can afford only a small number of sensors.
With their move into the new building, GAStech also introduced new security procedures, which staff members are not necessarily adopting consistently. Staff members are now required to wear proximity (prox) cards while in the building. The building is instrumented with passive prox card readers that cover individual building zones. The prox card zones do not generally correspond with the HVAC zones. When a prox card passes into a new zone, it is detected and recorded. Most, but not all, areas are still open to staff members even if they forget their prox cards. People are somewhat careless with their prox cards, but some diligent staff members will go to the security desk and pick up a new prox card if their old one is mislaid. As part of the deal to entice GAStech to move into this new building, the builders included a free robotic mail delivery system. This robot, nicknamed Rosie, travels the halls periodically, moving between floors in a specially designed chute. Rosie is equipped with a mobile prox sensor, which identifies the prox cards in the areas she travels through.
The Task
As an expert in visual analytics, you have been hired to help GAStech understand its operations data. In this assignment, you are given two weeks of building and prox sensor data. Can you use visual analytics to identify typical patterns of and issues of concern?
You will be asked to answer the following types of questions:
- What are the typical patterns in the prox card data? What does a typical day look like for GAStech employees?
- Describe up to ten of the most interesting patterns that appear in the building data. Describe what is notable about the pattern and explain its possible significance.
- Describe up to ten notable anomalies or unusual events you see in the data. Prioritize those issues that are most likely to represent a danger or a serious issue for building operations.
- Describe up to five observed relationships between the proximity card data and building data elements. If you find a causal relationship (for example, a building event or condition leading to personnel behavior changes or personnel activity leading to building operations changes), describe your discovered cause and effect, the evidence you found to support it, and your level of confidence in your assessment of the relationship.
Background
Heating, ventilation and air conditioning (HVAC) is the technology of indoor and vehicular environmental comfort. Its goal is to provide thermal comfort and acceptable indoor air quality.
Key Measurement attributes and their significance
- Lights Power [W] Power used by the lights in the zone
- Equipment Power [W] Power used by the electric equipment in the zone
- Thermostat Temp [C] Temperature of the air inside the zone
- Thermostat Heating Setpoint [C] Heating set point schedule for the zone
- Thermostat Cooling Setpoint [C] Cooling set point schedule for the zone
- VAV Availability Manager Night Cycle Control Status On/off status of the HVAC system during periods when the system is normally scheduled off. The night cycle manager cycles the HVAC system to maintain night and weekend set point temperatures.
- VAV_SYS SUPPLY FAN:Fan Power [W] Power used by the HVAC system fan
- BATH_EXHAUST:Fan Power [W] Power used by the bathroom exhaust fan
- VAV REHEAT Damper Position Position of the zone's air supply box damper. 1 corresponds to fully open, 0 corresponds to fully closed
- REHEAT COIL Power [W] Power used by the zone air supply box reheat coil
- VAV_SYS HEATING COIL Power [W] Power used by the HVAC system heating coil
- VAV_SYS Outdoor Air Flow Fraction Percentage of total air delivered by the HVAC system that is from the outside
- VAV_SYS Outdoor Air Mass Flow Rate[kg/s] Flow rate of outside air entering the HVAC system
- VAV_SYS COOLING COIL Power [W] Power used by the HVAC system cooling coil
- VAV_SYS AIR LOOP INLET Temperature [C] Mixed temperature of air returning to the HVAC system from all zones it serves
- VAV_SYS AIR LOOP INLET Mass Flow Rate [kg/s] Total flow rate of air returning to the HVAC system from all zones it serves
- VAV_SYS SUPPLY FAN OUTLET Temperature [C] Temperature of the air exiting the HVAC system fan
- VAV_SYS SUPPLY FAN OUTLET Mass Flow Rate [kg/s] Total flow rate of air delivered by the HVAC system fan to the zones it serves
- RETURN OUTLET CO2 Concentration [ppm] Concentration of C02 measured at the zone's return air grille
- SUPPLY INLET Temperature [C] Temperature of the air entering the zone from its air supply box
- SUPPLY INLET Mass Flow Rate[kg/s] Flow rate of the air entering the zone from its air supply box
- Mechanical Ventilation Mass Flow Rate [kg/s] Ventilation rate of the zone exhaust fan
- Hazium Concentration Something
- Drybulb Temperature [C] Drybulb temperature of the outside air
- Water Heater Tank Temperature [C] Temperature of the water inside the hot water heater
- Water Heater Gas Rate [W] Rate at which the water heater burns natural gas
- Supply Side Inlet Mass Flow Rate [kg/s] Flow rate of water entering the hot water heater
- Supply Side Inlet Temperature [C] Temperature of the water entering the hot water heater
- HVAC Electric Demand Power [W] Total power used by the building's HVAC system including coils, fans and pumps.
- Total Electric Demand Power [W] Total power used by the building
- Loop Temp Schedule Temperature set point of the hot water loop. This is the temperature at which hot water is delivered to hot water appliances and fixtures.
- Water Heater Setpoint Water heater set point temperature
- DELI-FAN Power [W] Power used by the deli exhaust fan
- Pump Power [W] Power used by the hot water system pump Water Heater
The data
You will have the following data and supporting information at your disposal:
- A building layout for the GAStech offices, including the maps of the prox zones and the HVAC zones
- A current list of employees, roles, and office assignments
- A description of the data formats and fields provided
- Proximity sensor data for each of the prox zone regions
- Proximity sensor data from Rosie the mobile robot
- HVAC sensor readings and status information from each of the building’s HVAC zones
- Hazium readings from four sensors.
The datasets above can be generally grouped into 2 different category
- Employee Proximity Card Data
- Building Data
Data Cleaning
Problem #1 | Building Data |
---|---|
Issue | The original building data provided is not "user-friendly". Tableau does not have difficulty reading it, but to plot the charts, it will pose a lot of issue. |
Solution |
Problem #2 | Building Data |
---|---|
Issue | The original data provided, concatenate information into the column itself. For example "F_1_Z_1:Lights Power", it is a column header by itself. The column header tells us that the reading is taken from floor 1, zone 1 and it is measuring the Lights Power. However such information is only understood by humans instead of business intelligence software like Tableau. Therefore, there is a need for us to transform into a more "software-friendly" form. |
Solution |
Problem #3 | Proximity Card Data |
---|---|
Issue | Example |
Solution |
Problem #4 | Building Data |
---|---|
Issue | Example |
Solution |
Problem #5 | Building Data |
---|---|
Issue | Example |
Solution |
Problem #6 | Building Data |
---|---|
Issue | Example |
Solution |
Final Excel Files
- bldg-MC2.csv Contains all the building related data (including hazium)
- bldg-MC2_mapping.csv Contains all the necessary mapping of the attributes in building data, so that Tableau can understand which floor and zone which the data was taken from.
- employee.csv All the employee related data.
- proxData_Merged.csv Contains the merged data from both fixed and mobile proximity sensor.
- proxMobileOut-MC2.csv Contains all the proximity card data that are recorded by the mobile sensor (Roise).
- proxOUt-MC2.csv Contains all the proximity card data that are recorded by the fixed sensor.
- proxOut-MC2_zoning_polygon.csv Contains all the polygon mapping of the zones for the fixed proximity sensor.
Visualisation
The visualization is based on the category of the data. The breakdown of the proposed visualization is as shown below.
- Homepage
- Building Data Explorer : Air Supply Controls / Water Supply Controls / Fan Controls / Coil Controls / Additional System Controls
- Employee Movement Explorer
- Variable Explorer
The original dataset is overwhelming. There are over 400 different columns. To make the analysis more meaningful, the data columns has to be group logically based on the purpose of the sensor/data point. I have grouped the data into 6 different categories, namely;
- Air Supply data
- Water Supply data
- Fan data
- Coil data
- Additional System data
- Employee Proximity Card data
The design of the visualization is based on the "Overview first, zoom and filter, then details-on-demand" (Shneiderman, 1996). Thus when the user uses the tool, first he/she will be on the homepage [Step 1]. Through the homepage, it provides an overview of the data exploratory functions available. It provides a summary group of all the available data. The user will then choose the data of his/her interest and be redirected to it.
Once you are redirected at to the dashboard after [Step 1], you are at [Step 2] now. Basically, at this stage, you are looking at the data which you are interested in. You can interact with the data, by using the filters. Hovering on the data point will provide you more details on the data.
If you are keen to find out more about the particular dataset/column, you can proceed to [Step 3] where you explore in finer details of the variable which you are looking at. At [Step 3] you proceed to the Variable Explorer. At this dashboard, you are given the ability to drill down the data into finer details. For example, breaking the data up by floor, zones, time etc.
Alternatively, using the Employee Proximity Card data can be either in [Step 2] or [Step 3]. When you are at this dashboard, you can explore the locations of the employees and determine its correlation with the other dataset.
[Step 1] Homepage |
---|
Purpose / Description The homepage is the landing page you will see when you use this Visualisation tool. The data explorery tools are all displayed on the homepage. This homepage makes use of the Tableau Dashboard and its action functions to enable interactivity. It is to serve as a "Home" panel for this visualisation and it would enable the user ease of navigation between the dashboards. |
Interactive Technique
|
[Step 2] Building Data Explorer : Air Supply Controls / Water Supply Controls / Fan Controls / Coil Controls / Additional System Controls |
---|
Purpose / Description The purpose of this dashboard is to give the user and overview of the data of the related controls. Within the HVAC system, there are a lot of intra-working sub-systems which help keep the entire HVAC system working. This dashboard groups all the related controls together and presents an overview of the data. This will allow the user to easier understand the sub-systems of the data. In general, I had grouped the date into 5 sub-systems:
The dashboard is logically designed for to ease usability. The layout as shown below. The dashboard starts with the navigation bar right at the top, followed by the title and description. After which are the filters which are specific for the dashboard. The individual charts then follow. Within the charts itself, it is descriptive by nature. It has its title and this description of what the data is trying to measure. |
Interactive Technique
|
Types of Charts used The data provided are readings taken from various HVAC/Proximity Sensors. Thus, all of the readings are taken against time. To do meaningful comparison and analysis with time as one of the dimension, I used mainly,
The image below is a representative of the type of charts used. It does not represent all the charts that are present in the dashboard. |
[Step 2/3] Employee Movement Explorer |
---|
Purpose / Description The purpose of this dashboard is visualise the employee proximity card data. The data are given with X,Y coordinates. Thus, we can plot the data on a background image map which is provided in the original dataset. The proximity card data are visualized on the floor map itself. There are modifications to the floor map so that the data can be better visualized. Now that the employee's movements are visualized on an image map, it gives much higher clarity on the employee's movement/activities around the building. |
Interactive Technique
|
Types of Charts used For this dashboard, much of the data are given based on the location itself. Thus, the data needs to be plotted on an image to effectively show the pattern between the employee's location and the time of the day. This will help to tell us what the employee's movement/activities are like.
The image below is a representative of the type of charts used. It does not represent all the charts that are present in the dashboard. |
[Step 3] Variable Explorer |
---|
Purpose / Description Variable Explorer is to allow the user to further explore the data in more details. In the previous dashboards, especially for the controls, the level of detail is limited so that the analyst can see the bigger picture. In this dashboard, it is designed to empower the analyst to view more about the data and how it changes across floor, zones and time. This is to help the analyst understand how the readings varies across the mentioned building attributes and time. The aim of this dashboard is to focus on just one measurement and understand its pattern/behaviour. |
Interactive Technique
|
Types of Charts used The data all have one common attribute, which is date/time. Thus, to enable flexibility for the dashboard to handle all of the variable types, the dashboard is fundamentally be required to visualize time-related data. Therefore, the following types of charts are used.
The image below is a representative of the type of charts used. It does not represent all the charts that are present in the dashboard. |
Use Case
Findings - Task #1
What are the typical patterns in the prox card data? What does a typical day look like for GAStech employees?
Findings - Task #2
Describe up to ten of the most interesting patterns that appear in the building data. Describe what is notable about the pattern and explain its possible significance.
Findings - Task #3
Describe up to ten notable anomalies or unusual events you see in the data. Prioritize those issues that are most likely to represent a danger or a serious issue for building operations.
Findings - Task #4
Describe up to five observed relationships between the proximity card data and building data elements. If you find a causal relationship (for example, a building event or condition leading to personnel behavior changes or personnel activity leading to building operations changes), describe your discovered cause and effect, the evidence you found to support it, and your level of confidence in your assessment of the relationship.
Conclusion
My initial work - https://public.tableau.com/views/MA_3_Final/Overview?:embed=y&:display_count=yes
Visualisation Software
To perform the visual analysis, students are encouraged to explore any one or a combination of the following software:
- Tableau
- JMP Pro
- Qlik Sense
- Microsoft Power BI
One of the goals of this assignment is for you to learn to use and evaluate the effectiveness of these visual analytics tools.
Submission details
This is an individual assignment. You are required to work on the assignment and prepare submission individually. Your completed assignment is due on 24th October 2016, by 12.00 noon.
You need to edit your assignment in the appropriate wiki page of the Assignment Dropbox. The title of the wiki page should be in the form of: IS428_2016-17_T1_Assign3_FullName.
The assignment 3 wiki page should include the URL link to the web-based interactive data visualization system prepared.
Assignment 3 Q&A
Need more clarification, please feel free to pen down your questions.
- What is Hazium? Hazium is a (fictitious) chemical that has become a recent concern on the island of Kronos. Not much is known about its effects, but it is suspected that Hazium is not good for people.
- There are a few extra building file data fields in the .json dataset that do not appear in the .csv data. These extra data fields are actually valid for the building for the dates and times they were recorded, but they will not add significantly to your analysis. So for this assignment, please just use the data fields included in the .csv file.
- Can you provide more info on the data provided in the mobile proximity card data? Are the x,y coordinates bound to a normal (x,y) plane, where in this case the plane is the floor maps? The (x,y) coordinates are bound to a normal plane. The (x,y) plus the floor number would identify a specific location. The lower left of the provided map is (0,0) and the upper right is (189,111).
- In some cases, data is reported for some sensors and not others, or it is documented but not reported. Where can we find this data? Please use the data fields you have available to perform your investigation. In general, the documented set of attributes may not be reported for all zones.
- What does the (x,y) coordinates represent for the mobile robot sensor? The (x,y) coordinates for these reading represent the location of the mobile sensor.
- Sometimes, mobile prox data for a prox card repeats multiple times in a minute. Does this indicate the number of seconds that the prox card was within range of the sensor? No. Multiple readings do not indicate what fraction of the minute that the mobile sensor was in proximity of the prox card.
- In some cases, the value of the VAV Availability Manager Night Cycle On/Off is 2. Is this a valid value? Yes.
- Does F_3_Z_9 VAV Damper Position mean F_3_Z_9 VAV REHEAT Damper Position? Yes.
References
- http://www.picturetopeople.org/image_utilities/image-grayscale-converter/grayscale-image-generator.html
- https://community.tableau.com/message/320738
- http://www.thedataschool.co.uk/niccolo-cirone/tableau-tip-week-wednesday-creating-dashboard-navigator-buttons/
- http://kb.tableau.com/articles/howto/renaming-dimension-column-row-headers
- https://tableauandbehold.com/2015/04/13/creating-custom-polygons-on-a-background-image/
- https://www.kane.co.uk/knowledge-centre/what-are-safe-levels-of-co-and-co2-in-rooms
- https://en.wikipedia.org/wiki/HVAC