IS480 Team wiki: 2017T1 Ravenous Analytics User Testing 1
Jump to navigation
Jump to search
Contents
Conclusion of Analytics Dashboard User Testing 1
Test Plan
Click here for test instructions
Goals Reached
S/N | Goals | Reached? | Remarks |
---|---|---|---|
1 | Participants should be able to accurately gather statistics for claim rate and active rate across all agencies and PSD | Yes | All participants managed to interpret the data correctly and entered the correct values |
2 | Participants should be able to correctly draw an inference from the 4 claim and active rate charts | Yes | All participants were able to compare data from their own department to all agencies correctly. They were able to identify that PSD had a higher claim rate as compared to all agencies |
3 | Participants should be able to gather the statistics for number of logins on PC and mobile for different dates | Yes | All participants managed to interpret the data correctly and entered the correct values |
4 | Participants should understand what the "Group Activity" graph is trying to express | Yes | 2 out of 5 participants gave the answer we were looking for. The other participants did not give the answer we were looking for, but were able to correctly infer what the graph was trying to depict |
5 | Participants should be able to gather statistic for number of posts, shares, comments and likes in previous week | Yes | All participants managed to interpret the data correctly and entered the correct values |
6 | Participants should be able to draw Inference from the table of posts, shares, comments and likes | Yes | 2 out of 5 participants gave the answer we were looking for. The other participants did not give the answer we were looking for, but were able to correctly infer what the graph was trying to depict |
7 | Participants should be able to identify what “Story” the overview dashboard is trying to express. | Yes | "This is a very broad question. 2 out of 5 participants were able to 100% describe what the dashboard was about. The rest were able to identify parts of the dashboard story. Generally, all participants were able to understand the story of the dashboard." |
8 | Participants should correctly infer what the chart and the table “Engagement Score Across PSD Group Charts” are trying to depict. | Yes | All participants were able to correctly infer that the graph is trying to show |
9 | Participants should correctly infer what the doughnut chart is trying to depict | Yes | "Most participants inferred the chart correctly. 1 participant inferred the chart wrongly." |
10 | Participants should correctly infer what the chart “Group Activity Chart” is trying to depict. | No | Most participants could not interpret the graph correctly. Some could not understand what the graph was trying to tell. |
11 | Participants should be able to correctly infer both the charts together and know what both the charts are trying to depict | Yes | "Most of the participants were able to infer that Workplace Mobile is used more than WorkChat Mobile. Although some participants did not give us the exact answer we were looking out for, they were still able to understand the chart" |
12 | Participants should be able to correctly infer what the chart “Login Analysis” is trying to depict. | No | "Most participants could not interpret the graph correctly. Some could not understand what the graph was trying to tell. Furthermore, one participant was confused if the login is used for workplace or workchat. This is of concern to us." |
13 | Participants should be able to identify the top 3 day-and-time users are actively posting and commenting on posts | Yes | "All participants managed to interpret the data correctly and entered the correct values. 1 participant misinterpreted the question" |
14 | Participants should be able to decide when is the best time to post an article on Workplace@FB | No | All 5 participants had different answers. None of the participants got the correct answer |
15 | Participants should be able to Infer what the chart “Content on Workplace” is trying to depict. | Yes | All participants were able to correctly infer that the graph is trying to depict |
16 | Participants should be able to infer what the chart “Word Cloud” is trying to depict. | Yes | "Most participants were able to understand what the word cloud is for. 1 participant was not able to figure out what the word cloud is for" |
17 | Participants should be able to list down the top 5 words that capture their attention. | No | Most participants had different answers. Answers were accurate up to top 7 words |
Key Findings
Function | Observations/Users' Comments | Changes to be made |
---|---|---|
Group Activity Chart - Group | The phrase "moving" was not intuitive for participants, so participants could not interpret the chart accurately | Rework the calculation of Group Activity Chart and the visualization of the chart. |
Login Analysis - People | The phrase "moving" was not intuitive for participants, so participants could not interpret the chart accurately | Rework the calculation of Login Analysis Chart and the visualization of the chart. |
Login Analysis - People | Participants were confused if the data refers to logins specific to Workplace or the WorkChat app | Explore helper tips to make the function clearer or, look at repositioning the chart to a more intuitive location |
Post-Time & Comment-Time Analysis Chart - Content | Non-intuitive naming of graph. | Come up with a more intuitive header for this chart |
Post-Time & Comment-Time Analysis Chart - Content | Graph UI is too small. Difficult for users to read at a glance. Poor color choice too. | Redesign/resize the graph for friendlier UI/UX |
Word Cloud - Content | Participants had trouble identifying the top 5 words used | Redesign word cloud UI - No overlap |
Overall results/comments
> Improvements can be made to User Interface in terms of sizing, color choice and spacing between charts.
> Participants suggested placing a short description of the chart in the form of a helper tip beside the chart header.
> Participants gave an average of 5.2 out of 6 in terms of usefulness of the charts and all participants were able to interpret 90% of the charts and graphs accurately without any prompts or helper descriptions. Thus, we still consider UAT 1 to be a success
Full results can be found here