HeaderSIS.jpg

IS480 Team wiki: 2017T1 Ravenous Analytics User Testing 1

From IS480
Jump to navigation Jump to search
Ravenouslogo.png


Iconh1.png
Iconau1.png
Iconpo1.png
Iconpm1.png
Icond.png

 

Ravenousinternaltest.png Ravenousuat12.png Ravenousuat2.png Ravenousuat3.png Ravenousuat4.png


Ravenousclick.png Ravenousanalytics1.png Ravenousevbot.png Ravenousfabot.png

Conclusion of Analytics Dashboard User Testing 1

Test Plan

Click here for test instructions

Goals Reached

S/N Goals Reached? Remarks
1 Participants should be able to accurately gather statistics for claim rate and active rate across all agencies and PSD Yes All participants managed to interpret the data correctly and entered the correct values
2 Participants should be able to correctly draw an inference from the 4 claim and active rate charts Yes All participants were able to compare data from their own department to all agencies correctly. They were able to identify that PSD had a higher claim rate as compared to all agencies
3 Participants should be able to gather the statistics for number of logins on PC and mobile for different dates Yes All participants managed to interpret the data correctly and entered the correct values
4 Participants should understand what the "Group Activity" graph is trying to express Yes 2 out of 5 participants gave the answer we were looking for. The other participants did not give the answer we were looking for, but were able to correctly infer what the graph was trying to depict
5 Participants should be able to gather statistic for number of posts, shares, comments and likes in previous week Yes All participants managed to interpret the data correctly and entered the correct values
6 Participants should be able to draw Inference from the table of posts, shares, comments and likes Yes 2 out of 5 participants gave the answer we were looking for. The other participants did not give the answer we were looking for, but were able to correctly infer what the graph was trying to depict
7 Participants should be able to identify what “Story” the overview dashboard is trying to express. Yes "This is a very broad question. 2 out of 5 participants were able to 100% describe what the dashboard was about. The rest were able to identify parts of the dashboard story. Generally, all participants were able to understand the story of the dashboard."
8 Participants should correctly infer what the chart and the table “Engagement Score Across PSD Group Charts” are trying to depict. Yes All participants were able to correctly infer that the graph is trying to show
9 Participants should correctly infer what the doughnut chart is trying to depict Yes "Most participants inferred the chart correctly. 1 participant inferred the chart wrongly."
10 Participants should correctly infer what the chart “Group Activity Chart” is trying to depict. No Most participants could not interpret the graph correctly. Some could not understand what the graph was trying to tell.
11 Participants should be able to correctly infer both the charts together and know what both the charts are trying to depict Yes "Most of the participants were able to infer that Workplace Mobile is used more than WorkChat Mobile. Although some participants did not give us the exact answer we were looking out for, they were still able to understand the chart"
12 Participants should be able to correctly infer what the chart “Login Analysis” is trying to depict. No "Most participants could not interpret the graph correctly. Some could not understand what the graph was trying to tell. Furthermore, one participant was confused if the login is used for workplace or workchat. This is of concern to us."
13 Participants should be able to identify the top 3 day-and-time users are actively posting and commenting on posts Yes "All participants managed to interpret the data correctly and entered the correct values. 1 participant misinterpreted the question"
14 Participants should be able to decide when is the best time to post an article on Workplace@FB No All 5 participants had different answers. None of the participants got the correct answer
15 Participants should be able to Infer what the chart “Content on Workplace” is trying to depict. Yes All participants were able to correctly infer that the graph is trying to depict
16 Participants should be able to infer what the chart “Word Cloud” is trying to depict. Yes "Most participants were able to understand what the word cloud is for. 1 participant was not able to figure out what the word cloud is for"
17 Participants should be able to list down the top 5 words that capture their attention. No Most participants had different answers. Answers were accurate up to top 7 words

Key Findings

Function Observations/Users' Comments Changes to be made
Group Activity Chart - Group The phrase "moving" was not intuitive for participants, so participants could not interpret the chart accurately Rework the calculation of Group Activity Chart and the visualization of the chart.
Login Analysis - People The phrase "moving" was not intuitive for participants, so participants could not interpret the chart accurately Rework the calculation of Login Analysis Chart and the visualization of the chart.
Login Analysis - People Participants were confused if the data refers to logins specific to Workplace or the WorkChat app Explore helper tips to make the function clearer or, look at repositioning the chart to a more intuitive location
Post-Time & Comment-Time Analysis Chart - Content Non-intuitive naming of graph. Come up with a more intuitive header for this chart
Post-Time & Comment-Time Analysis Chart - Content Graph UI is too small. Difficult for users to read at a glance. Poor color choice too. Redesign/resize the graph for friendlier UI/UX
Word Cloud - Content Participants had trouble identifying the top 5 words used Redesign word cloud UI

- No overlap
- Shade/Coloring

Overall results/comments

> Improvements can be made to User Interface in terms of sizing, color choice and spacing between charts.
> Participants suggested placing a short description of the chart in the form of a helper tip beside the chart header.
> Participants gave an average of 5.2 out of 6 in terms of usefulness of the charts and all participants were able to interpret 90% of the charts and graphs accurately without any prompts or helper descriptions. Thus, we still consider UAT 1 to be a success

Full results can be found here