Difference between revisions of "IS480 Team wiki: 2010T1 The Green Reapers: Usability Testing"
(4 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
− | =<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='# | + | '''[[IS480_Team_wiki:_2010T1_The_Green_Reapers|Back to Home Page]]''' |
+ | |||
+ | To complement functional testing with our mentors at IRRI, we also pushed hard to improve the usability of our features. We reached out to untrained users to see how well they might take to the functions without any prompting. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | =<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#eaf7ff', endColorstr='#b9e6ff'); background: -webkit-gradient(linear, left top, left bottom, from(#eaf7ff), to(#b9e6ff)); background: -moz-linear-gradient(top, #eaf7ff, #b9e6ff); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em"> Methodology </div>= | ||
<ul style="line-height: 180%;"> | <ul style="line-height: 180%;"> | ||
− | <li>2 rounds of testing</li> | + | <li>2 rounds of usability testing</li> |
+ | <ul style="line-height: 180%;"> | ||
+ | <li>Iteration 6: 24 - 26 October</li> | ||
+ | <li>Iteration 7: 12 - 15 November</li> | ||
+ | </ul> | ||
<li>Task-based tests: | <li>Task-based tests: | ||
<ul style="line-height: 180%;"> | <ul style="line-height: 180%;"> | ||
Line 23: | Line 34: | ||
− | =<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='# | + | =<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#eaf7ff', endColorstr='#b9e6ff'); background: -webkit-gradient(linear, left top, left bottom, from(#eaf7ff), to(#b9e6ff)); background: -moz-linear-gradient(top, #eaf7ff, #b9e6ff); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em"> Test Questions </div>= |
+ | |||
== Analytics == | == Analytics == | ||
+ | |||
+ | {| class="wikitable" style="margin: auto" | ||
+ | ! colspan="3" style="background: black; color: white" | IntelliCrops Feedback - Analytics | ||
+ | |- | ||
+ | | colspan="2" | Please head over to http://www.intellicrops.com/analytics/ and create 3 different charts that show | ||
+ | |||
+ | 1) the total number of calls that come from each country (pie) | ||
+ | 2) the different average monthly call durations of each country (clustered columns) | ||
+ | 3) the different average daily call durations of each country (clustered columns) | ||
+ | | rowspan="7" | [[image:IntelliCrops-Usability-Analytics-Form.png|200px]] | ||
+ | |- | ||
+ | ! Question | ||
+ | ! Type | ||
+ | |- | ||
+ | | Were you able to create the graphs successfully? | ||
+ | | Yes / No | ||
+ | |- | ||
+ | | How easy was it to create the graphs? | ||
+ | | Scale of '''(1) difficult''' - '''(7) easy''' | ||
+ | |- | ||
+ | | How easy was it to view the graphs? | ||
+ | | Scale of '''(1) bad''' - '''(7) good''' | ||
+ | |- | ||
+ | | Did you encounter any problems while using the IVR System? | ||
+ | | Textarea | ||
+ | |- | ||
+ | | Any other comments | ||
+ | | Textarea | ||
+ | |} | ||
+ | |||
== Interactive Voice Response == | == Interactive Voice Response == | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="3" | IntelliCrops Feedback - Interactive Voice Response | + | ! colspan="3" style="background: black; color: white" | IntelliCrops Feedback - Interactive Voice Response |
|- | |- | ||
| colspan="2" | (Requires a Skype account) | | colspan="2" | (Requires a Skype account) | ||
Line 41: | Line 83: | ||
2) Get a diagnosis for poor grain germination | 2) Get a diagnosis for poor grain germination | ||
− | |rowspan="8" | [[image:IntelliCrops-Usability-IVR-Form.png|200px]] | + | | rowspan="8" | [[image:IntelliCrops-Usability-IVR-Form.png|200px]] |
|- | |- | ||
! Question | ! Question | ||
Line 67: | Line 109: | ||
== Knowledge Navigator == | == Knowledge Navigator == | ||
+ | |||
+ | {| class="wikitable" style="margin: auto" | ||
+ | ! colspan="3" style="background: black; color: white" | IntelliCrops Feedback - Knowledge Navigator | ||
+ | |- | ||
+ | | colspan="2" | Please head over to http://www.intellicrops.com/ivr/knowledgenavigator.php | ||
+ | |||
+ | 1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice | ||
+ | |||
+ | 2) Get a diagnosis for poor grain germination | ||
+ | | rowspan="6" | [[image:IntelliCrops-Usability-KN-Form.png|200px]] | ||
+ | |- | ||
+ | ! Question | ||
+ | ! Type | ||
+ | |- | ||
+ | | Were you able to find the answers using the Knowledge Navigator? | ||
+ | | Yes / No | ||
+ | |- | ||
+ | | How easy was it to use the Knowledge Navigator? | ||
+ | | Scale of '''(1) difficult''' - '''(7) easy''' | ||
+ | |- | ||
+ | | Did you encounter any problems while using the Knowledge Navigator? | ||
+ | | Textarea | ||
+ | |- | ||
+ | | Any other comments | ||
+ | | Textarea | ||
+ | |} | ||
Line 73: | Line 141: | ||
− | =<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='# | + | =<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#eaf7ff', endColorstr='#b9e6ff'); background: -webkit-gradient(linear, left top, left bottom, from(#eaf7ff), to(#b9e6ff)); background: -moz-linear-gradient(top, #eaf7ff, #b9e6ff); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em"> Results and Action Plan </div>= |
== Analytics == | == Analytics == | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="3" | Usability Testing Round 1 | + | ! colspan="3" style="background: black; color: white" | Usability Testing Round 1 |
|- | |- | ||
| [[Image:IntelliCrops-Usability-Round_1-Analytics-Complete.png|230px]] | | [[Image:IntelliCrops-Usability-Round_1-Analytics-Complete.png|230px]] | ||
Line 87: | Line 155: | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="3" | Usability Testing Round 2 | + | ! colspan="3" style="background: black; color: white" | Improved Results Observed During Usability Testing Round 2 |
|- | |- | ||
| [[Image:IntelliCrops-Usability-Round_2-Analytics-Complete.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-Analytics-Complete.png|230px]] | ||
Line 93: | Line 161: | ||
| [[Image:IntelliCrops-Usability-Round_2-Analytics-View.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-Analytics-View.png|230px]] | ||
|} | |} | ||
+ | |||
+ | |||
+ | {| class="wikitable" style="margin: auto" | ||
+ | ! colspan="2" style="background: black; color: white" | Discussion | ||
+ | |- | ||
+ | ! Issue | ||
+ | ! Fix | ||
+ | |- | ||
+ | | Labels and values vague and ambiguous. | ||
+ | |<ul style="line-height: 180%;"> | ||
+ | <li>We previously assumed some familiarity with graphical analysis.</li> | ||
+ | <li>We've added hints and labelled previews to describe the nature and impact of the various fields</li> | ||
+ | </ul> | ||
+ | |} | ||
+ | |||
+ | |||
+ | |||
+ | |||
Line 98: | Line 184: | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="3" | Usability Testing Round | + | ! colspan="3" style="background: black; color: white" | Usability Testing Round 2 |
|- | |- | ||
| [[Image:IntelliCrops-Usability-Round_1-IVR-Completion.png|230px]] | | [[Image:IntelliCrops-Usability-Round_1-IVR-Completion.png|230px]] | ||
Line 107: | Line 193: | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="3" | Usability Testing Round 2 | + | ! colspan="3" style="background: black; color: white" | Improved Results Observed During Usability Testing Round 2 |
|- | |- | ||
| [[Image:IntelliCrops-Usability-Round_2-IVR-Completion.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-IVR-Completion.png|230px]] | ||
| [[Image:IntelliCrops-Usability-Round_2-IVR-Easy.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-IVR-Easy.png|230px]] | ||
| [[Image:IntelliCrops-Usability-Round_2-IVR-Quality.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-IVR-Quality.png|230px]] | ||
+ | |} | ||
+ | |||
+ | |||
+ | {| class="wikitable" style="margin: auto" | ||
+ | ! colspan="2" style="background: black; color: white" | Discussion | ||
+ | |- | ||
+ | ! Issue | ||
+ | ! Fix | ||
+ | |- | ||
+ | | Pressing the numbers didn’t seem to work | ||
+ | |<ul style="line-height: 180%;"> | ||
+ | <li>We assumed that most people have had experience with other interactive voice response systems, such that pressing hex after their option would be natural.</li> | ||
+ | <li>However, some participants neglected to do so, and were left wondering why the numbers didn’t work.</li> | ||
+ | <li>We solved this by adding more helpful user prompts. For example, we now ask user to press the number, and “hex”.</li> | ||
+ | |- | ||
+ | | Poor help provided | ||
+ | |<ul style="line-height: 180%;"> | ||
+ | <li>When we interviewed the participants, we found that poor help referred to the inability to repeat options or return to the main menu. We added this functionality before the second round of tests.</li></ul> | ||
+ | |- | ||
+ | | Cannot really hear the menu options. Quite muffled | ||
+ | |<ul style="line-height: 180%;"> | ||
+ | <li>We investigated and found a few parameters to consider – network speed was certainly an issue, but it was out of our hands, so we focused on tweaking the speed of the synthesized voice, and added extra punctuation to accentuate the various points of emphasis.</li></ul> | ||
|} | |} | ||
+ | |||
+ | |||
+ | |||
Line 119: | Line 230: | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="2" | Usability Testing Round 1 | + | ! colspan="2" style="background: black; color: white" | Usability Testing Round 1 |
|- | |- | ||
| [[Image:IntelliCrops-Usability-Round_1-KN-Complete.png|230px]] | | [[Image:IntelliCrops-Usability-Round_1-KN-Complete.png|230px]] | ||
Line 127: | Line 238: | ||
{| class="wikitable" style="margin: auto" | {| class="wikitable" style="margin: auto" | ||
− | ! colspan="2" | Usability Testing Round 2 | + | ! colspan="2" style="background: black; color: white" | Improved Results Observed During Usability Testing Round 2 |
|- | |- | ||
| [[Image:IntelliCrops-Usability-Round_2-KN-Complete.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-KN-Complete.png|230px]] | ||
| [[Image:IntelliCrops-Usability-Round_2-KN-Easy.png|230px]] | | [[Image:IntelliCrops-Usability-Round_2-KN-Easy.png|230px]] | ||
|} | |} | ||
+ | |||
+ | |||
+ | {| class="wikitable" style="margin: auto" | ||
+ | ! colspan="2" style="background: black; color: white" | Discussion | ||
+ | |- | ||
+ | ! Issue | ||
+ | ! Fix | ||
+ | |- | ||
+ | | Browser-specific display bug that prevented the content from showing up | ||
+ | |<ul style="line-height: 180%;"> | ||
+ | <li>Fixed; now cross-browser</li> | ||
+ | |- | ||
+ | | Content displayed all bunched-up together | ||
+ | |<ul style="line-height: 180%;"> | ||
+ | <li>Added content detection to add breaks and paragraphing at suitable locations</li></ul> | ||
+ | |} | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | '''[[IS480_Team_wiki:_2010T1_The_Green_Reapers|Back to Home Page]]''' |
Latest revision as of 15:46, 28 November 2010
To complement functional testing with our mentors at IRRI, we also pushed hard to improve the usability of our features. We reached out to untrained users to see how well they might take to the functions without any prompting.
Contents
Methodology
- 2 rounds of usability testing
- Iteration 6: 24 - 26 October
- Iteration 7: 12 - 15 November
- Task-based tests:
- Unassisted as far as possible
- Model real-life task goals
- Focused on usability & discoverability
- Mixed background of users
- 75% non-technical
- Mix of male and female, young and old
Test Questions
Analytics
IntelliCrops Feedback - Analytics | ||
---|---|---|
Please head over to http://www.intellicrops.com/analytics/ and create 3 different charts that show
1) the total number of calls that come from each country (pie) 2) the different average monthly call durations of each country (clustered columns) 3) the different average daily call durations of each country (clustered columns) |
||
Question | Type | |
Were you able to create the graphs successfully? | Yes / No | |
How easy was it to create the graphs? | Scale of (1) difficult - (7) easy | |
How easy was it to view the graphs? | Scale of (1) bad - (7) good | |
Did you encounter any problems while using the IVR System? | Textarea | |
Any other comments | Textarea |
Interactive Voice Response
IntelliCrops Feedback - Interactive Voice Response | ||
---|---|---|
(Requires a Skype account)
Please head over to http://www.intellicrops.com/wordpress/ and click on the Skype icon to start dialing 1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice Press 0# to return to the main menu 2) Get a diagnosis for poor grain germination |
||
Question | Type | |
Were you able to find the answers using the IVR system? | Yes / No | |
Did you encounter any problems while using the IVR System? | Textarea | |
How easy was it to use the IVR? | Scale of (1) difficult - (7) easy | |
How was the quality of the voice? | Scale of (1) bad - (7) good | |
Were the answers too lengthy? | Yes / No | |
Any other comments | Textarea |
IntelliCrops Feedback - Knowledge Navigator | ||
---|---|---|
Please head over to http://www.intellicrops.com/ivr/knowledgenavigator.php
1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice 2) Get a diagnosis for poor grain germination |
||
Question | Type | |
Were you able to find the answers using the Knowledge Navigator? | Yes / No | |
How easy was it to use the Knowledge Navigator? | Scale of (1) difficult - (7) easy | |
Did you encounter any problems while using the Knowledge Navigator? | Textarea | |
Any other comments | Textarea |
Results and Action Plan
Analytics
Usability Testing Round 1 | ||
---|---|---|
Improved Results Observed During Usability Testing Round 2 | ||
---|---|---|
Discussion | |
---|---|
Issue | Fix |
Labels and values vague and ambiguous. |
|
Interactive Voice Response
Usability Testing Round 2 | ||
---|---|---|
Improved Results Observed During Usability Testing Round 2 | ||
---|---|---|
Discussion | |
---|---|
Issue | Fix |
Pressing the numbers didn’t seem to work |
|
Poor help provided |
|
Cannot really hear the menu options. Quite muffled |
|
Usability Testing Round 1 | |
---|---|
Improved Results Observed During Usability Testing Round 2 | |
---|---|
Discussion | |
---|---|
Issue | Fix |
Browser-specific display bug that prevented the content from showing up |
|
Content displayed all bunched-up together |
|