HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2010T1 The Green Reapers: Usability Testing"

From IS480
Jump to navigation Jump to search
 
(4 intermediate revisions by one other user not shown)
Line 1: Line 1:
=<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#f6ffe9', endColorstr='#bfff64'); background: -webkit-gradient(linear, left top, left bottom, from(#f6ffe9), to(#bfff64)); background: -moz-linear-gradient(top,  #f6ffe9,  #bfff64); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em">      Methodology      </div>=
+
'''[[IS480_Team_wiki:_2010T1_The_Green_Reapers|Back to Home Page]]'''
 +
 
 +
To complement functional testing with our mentors at IRRI, we also pushed hard to improve the usability of our features.  We reached out to untrained users to see how well they might take to the functions without any prompting.
 +
 
 +
 
 +
 
 +
 
 +
=<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#eaf7ff', endColorstr='#b9e6ff'); background: -webkit-gradient(linear, left top, left bottom, from(#eaf7ff), to(#b9e6ff)); background: -moz-linear-gradient(top,  #eaf7ff,  #b9e6ff); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em">      Methodology      </div>=
  
 
<ul style="line-height: 180%;">
 
<ul style="line-height: 180%;">
<li>2 rounds of testing</li>
+
<li>2 rounds of usability testing</li>
 +
<ul style="line-height: 180%;">
 +
<li>Iteration 6: 24 - 26 October</li>
 +
<li>Iteration 7: 12 - 15 November</li>
 +
</ul>
 
<li>Task-based tests:
 
<li>Task-based tests:
 
<ul style="line-height: 180%;">
 
<ul style="line-height: 180%;">
Line 23: Line 34:
  
  
=<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#f6ffe9', endColorstr='#bfff64'); background: -webkit-gradient(linear, left top, left bottom, from(#f6ffe9), to(#bfff64)); background: -moz-linear-gradient(top,  #f6ffe9,  #bfff64); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em">      Test Questions      </div>=
+
=<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#eaf7ff', endColorstr='#b9e6ff'); background: -webkit-gradient(linear, left top, left bottom, from(#eaf7ff), to(#b9e6ff)); background: -moz-linear-gradient(top,  #eaf7ff,  #b9e6ff); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em">      Test Questions      </div>=
 +
 
  
 
== Analytics ==
 
== Analytics ==
 +
 +
{| class="wikitable" style="margin: auto"
 +
! colspan="3" style="background: black; color: white" | IntelliCrops Feedback - Analytics
 +
|-
 +
| colspan="2" | Please head over to http://www.intellicrops.com/analytics/ and create 3 different charts that show
 +
 +
1) the total number of calls that come from each country (pie)
 +
2) the different average monthly call durations of each country (clustered columns)
 +
3) the different average daily call durations of each country (clustered columns)
 +
| rowspan="7" | [[image:IntelliCrops-Usability-Analytics-Form.png|200px]]
 +
|-
 +
! Question
 +
! Type
 +
|-
 +
| Were you able to create the graphs successfully?
 +
| Yes / No
 +
|-
 +
| How easy was it to create the graphs?
 +
| Scale of '''(1) difficult''' - '''(7) easy'''
 +
|-
 +
| How easy was it to view the graphs?
 +
| Scale of '''(1) bad''' - '''(7) good'''
 +
|-
 +
| Did you encounter any problems while using the IVR System?
 +
| Textarea
 +
|-
 +
| Any other comments
 +
| Textarea
 +
|}
 +
  
 
== Interactive Voice Response ==
 
== Interactive Voice Response ==
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="3" | IntelliCrops Feedback - Interactive Voice Response
+
! colspan="3" style="background: black; color: white" | IntelliCrops Feedback - Interactive Voice Response
 
|-
 
|-
 
| colspan="2" | (Requires a Skype account)
 
| colspan="2" | (Requires a Skype account)
Line 41: Line 83:
  
 
2) Get a diagnosis for poor grain germination
 
2) Get a diagnosis for poor grain germination
|rowspan="8" | [[image:IntelliCrops-Usability-IVR-Form.png|200px]]
+
| rowspan="8" | [[image:IntelliCrops-Usability-IVR-Form.png|200px]]
 
|-
 
|-
 
! Question
 
! Question
Line 67: Line 109:
  
 
== Knowledge Navigator ==
 
== Knowledge Navigator ==
 +
 +
{| class="wikitable" style="margin: auto"
 +
! colspan="3" style="background: black; color: white" | IntelliCrops Feedback - Knowledge Navigator
 +
|-
 +
| colspan="2" | Please head over to http://www.intellicrops.com/ivr/knowledgenavigator.php
 +
 +
1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice
 +
 +
2) Get a diagnosis for poor grain germination
 +
| rowspan="6" | [[image:IntelliCrops-Usability-KN-Form.png|200px]]
 +
|-
 +
! Question
 +
! Type
 +
|-
 +
| Were you able to find the answers using the Knowledge Navigator?
 +
| Yes / No
 +
|-
 +
| How easy was it to use the Knowledge Navigator?
 +
| Scale of '''(1) difficult''' - '''(7) easy'''
 +
|-
 +
| Did you encounter any problems while using the Knowledge Navigator?
 +
| Textarea
 +
|-
 +
| Any other comments
 +
| Textarea
 +
|}
  
  
Line 73: Line 141:
  
  
=<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#f6ffe9', endColorstr='#bfff64'); background: -webkit-gradient(linear, left top, left bottom, from(#f6ffe9), to(#bfff64)); background: -moz-linear-gradient(top,  #f6ffe9,  #bfff64); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em">      Results and Action Plan      </div>=
+
=<div style="width: 95%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#eaf7ff', endColorstr='#b9e6ff'); background: -webkit-gradient(linear, left top, left bottom, from(#eaf7ff), to(#b9e6ff)); background: -moz-linear-gradient(top,  #eaf7ff,  #b9e6ff); padding: 15px; font-weight: bold; margin-top: 35px; line-height: 1.4em">      Results and Action Plan      </div>=
  
 
== Analytics ==
 
== Analytics ==
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="3" | Usability Testing Round 1
+
! colspan="3" style="background: black; color: white" | Usability Testing Round 1
 
|-
 
|-
 
| [[Image:IntelliCrops-Usability-Round_1-Analytics-Complete.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_1-Analytics-Complete.png‎|230px]]
Line 87: Line 155:
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="3" | Usability Testing Round 2
+
! colspan="3" style="background: black; color: white" | Improved Results Observed During Usability Testing Round 2
 
|-
 
|-
 
| [[Image:IntelliCrops-Usability-Round_2-Analytics-Complete.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-Analytics-Complete.png‎|230px]]
Line 93: Line 161:
 
| [[Image:IntelliCrops-Usability-Round_2-Analytics-View.png|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-Analytics-View.png|230px]]
 
|}
 
|}
 +
 +
 +
{| class="wikitable" style="margin: auto"
 +
! colspan="2" style="background: black; color: white" | Discussion
 +
|-
 +
! Issue
 +
! Fix
 +
|-
 +
| Labels and values vague and ambiguous.
 +
|<ul style="line-height: 180%;">
 +
<li>We previously assumed some familiarity with graphical analysis.</li>
 +
<li>We've added hints and labelled previews to describe the nature and impact of the various fields</li>
 +
</ul>
 +
|}
 +
 +
 +
 +
  
  
Line 98: Line 184:
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="3" | Usability Testing Round 1
+
! colspan="3" style="background: black; color: white" | Usability Testing Round 2
 
|-
 
|-
 
| [[Image:IntelliCrops-Usability-Round_1-IVR-Completion.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_1-IVR-Completion.png‎|230px]]
Line 107: Line 193:
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="3" | Usability Testing Round 2
+
! colspan="3" style="background: black; color: white" | Improved Results Observed During Usability Testing Round 2
 
|-
 
|-
 
| [[Image:IntelliCrops-Usability-Round_2-IVR-Completion.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-IVR-Completion.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-IVR-Easy.png|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-IVR-Easy.png|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-IVR-Quality.png|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-IVR-Quality.png|230px]]
 +
|}
 +
 +
 +
{| class="wikitable" style="margin: auto"
 +
! colspan="2" style="background: black; color: white" | Discussion
 +
|-
 +
! Issue
 +
! Fix
 +
|-
 +
| Pressing the numbers didn’t seem to work
 +
|<ul style="line-height: 180%;">
 +
<li>We assumed that most people have had experience with other interactive voice response systems, such that pressing hex after their option would be natural.</li>
 +
<li>However, some participants neglected to do so, and were left wondering why the numbers didn’t work.</li>
 +
<li>We solved this by adding more helpful user prompts.  For example, we now ask user to press the number, and “hex”.</li>
 +
|-
 +
| Poor help provided
 +
|<ul style="line-height: 180%;">
 +
<li>When we interviewed the participants, we found that poor help referred to the inability to repeat options or return to the main menu.  We added this functionality before the second round of tests.</li></ul>
 +
|-
 +
| Cannot really hear the menu options. Quite muffled
 +
|<ul style="line-height: 180%;">
 +
<li>We investigated and found a few parameters to consider – network speed was certainly an issue, but it was out of our hands, so we focused on tweaking the speed of the synthesized voice, and added extra punctuation to accentuate the various points of emphasis.</li></ul>
 
|}
 
|}
 +
 +
 +
  
  
Line 119: Line 230:
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="2" | Usability Testing Round 1
+
! colspan="2" style="background: black; color: white" | Usability Testing Round 1
 
|-
 
|-
 
| [[Image:IntelliCrops-Usability-Round_1-KN-Complete.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_1-KN-Complete.png‎|230px]]
Line 127: Line 238:
  
 
{| class="wikitable" style="margin: auto"
 
{| class="wikitable" style="margin: auto"
! colspan="2" | Usability Testing Round 2
+
! colspan="2" style="background: black; color: white" | Improved Results Observed During Usability Testing Round 2
 
|-
 
|-
 
| [[Image:IntelliCrops-Usability-Round_2-KN-Complete.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-KN-Complete.png‎|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-KN-Easy.png|230px]]
 
| [[Image:IntelliCrops-Usability-Round_2-KN-Easy.png|230px]]
 
|}
 
|}
 +
 +
 +
{| class="wikitable" style="margin: auto"
 +
! colspan="2" style="background: black; color: white" | Discussion
 +
|-
 +
! Issue
 +
! Fix
 +
|-
 +
| Browser-specific display bug that prevented the content from showing up
 +
|<ul style="line-height: 180%;">
 +
<li>Fixed; now cross-browser</li>
 +
|-
 +
| Content displayed all bunched-up together
 +
|<ul style="line-height: 180%;">
 +
<li>Added content detection to add breaks and paragraphing at suitable locations</li></ul>
 +
|}
 +
 +
 +
 +
 +
'''[[IS480_Team_wiki:_2010T1_The_Green_Reapers|Back to Home Page]]'''

Latest revision as of 15:46, 28 November 2010

Back to Home Page

To complement functional testing with our mentors at IRRI, we also pushed hard to improve the usability of our features. We reached out to untrained users to see how well they might take to the functions without any prompting.



Methodology

  • 2 rounds of usability testing
    • Iteration 6: 24 - 26 October
    • Iteration 7: 12 - 15 November
  • Task-based tests:
    • Unassisted as far as possible
    • Model real-life task goals
    • Focused on usability & discoverability
  • Mixed background of users
    • 75% non-technical
    • Mix of male and female, young and old




Test Questions

Analytics

IntelliCrops Feedback - Analytics
Please head over to http://www.intellicrops.com/analytics/ and create 3 different charts that show

1) the total number of calls that come from each country (pie) 2) the different average monthly call durations of each country (clustered columns) 3) the different average daily call durations of each country (clustered columns)

IntelliCrops-Usability-Analytics-Form.png
Question Type
Were you able to create the graphs successfully? Yes / No
How easy was it to create the graphs? Scale of (1) difficult - (7) easy
How easy was it to view the graphs? Scale of (1) bad - (7) good
Did you encounter any problems while using the IVR System? Textarea
Any other comments Textarea


Interactive Voice Response

IntelliCrops Feedback - Interactive Voice Response
(Requires a Skype account)

Please head over to http://www.intellicrops.com/wordpress/ and click on the Skype icon to start dialing

1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice

Press 0# to return to the main menu

2) Get a diagnosis for poor grain germination

IntelliCrops-Usability-IVR-Form.png
Question Type
Were you able to find the answers using the IVR system? Yes / No
Did you encounter any problems while using the IVR System? Textarea
How easy was it to use the IVR? Scale of (1) difficult - (7) easy
How was the quality of the voice? Scale of (1) bad - (7) good
Were the answers too lengthy? Yes / No
Any other comments Textarea


Knowledge Navigator

IntelliCrops Feedback - Knowledge Navigator
Please head over to http://www.intellicrops.com/ivr/knowledgenavigator.php

1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice

2) Get a diagnosis for poor grain germination

IntelliCrops-Usability-KN-Form.png
Question Type
Were you able to find the answers using the Knowledge Navigator? Yes / No
How easy was it to use the Knowledge Navigator? Scale of (1) difficult - (7) easy
Did you encounter any problems while using the Knowledge Navigator? Textarea
Any other comments Textarea




Results and Action Plan

Analytics

Usability Testing Round 1
IntelliCrops-Usability-Round 1-Analytics-Complete.png IntelliCrops-Usability-Round 1-Analytics-Easy.png IntelliCrops-Usability-Round 1-Analytics-View.png


Improved Results Observed During Usability Testing Round 2
IntelliCrops-Usability-Round 2-Analytics-Complete.png IntelliCrops-Usability-Round 2-Analytics-Easy.png IntelliCrops-Usability-Round 2-Analytics-View.png


Discussion
Issue Fix
Labels and values vague and ambiguous.
  • We previously assumed some familiarity with graphical analysis.
  • We've added hints and labelled previews to describe the nature and impact of the various fields




Interactive Voice Response

Usability Testing Round 2
IntelliCrops-Usability-Round 1-IVR-Completion.png IntelliCrops-Usability-Round 1-IVR-Easy.png IntelliCrops-Usability-Round 1-IVR-Quality.png


Improved Results Observed During Usability Testing Round 2
IntelliCrops-Usability-Round 2-IVR-Completion.png IntelliCrops-Usability-Round 2-IVR-Easy.png IntelliCrops-Usability-Round 2-IVR-Quality.png


Discussion
Issue Fix
Pressing the numbers didn’t seem to work
  • We assumed that most people have had experience with other interactive voice response systems, such that pressing hex after their option would be natural.
  • However, some participants neglected to do so, and were left wondering why the numbers didn’t work.
  • We solved this by adding more helpful user prompts. For example, we now ask user to press the number, and “hex”.
Poor help provided
  • When we interviewed the participants, we found that poor help referred to the inability to repeat options or return to the main menu. We added this functionality before the second round of tests.
Cannot really hear the menu options. Quite muffled
  • We investigated and found a few parameters to consider – network speed was certainly an issue, but it was out of our hands, so we focused on tweaking the speed of the synthesized voice, and added extra punctuation to accentuate the various points of emphasis.




Knowledge Navigator

Usability Testing Round 1
IntelliCrops-Usability-Round 1-KN-Complete.png IntelliCrops-Usability-Round 1-KN-Easy.png


Improved Results Observed During Usability Testing Round 2
IntelliCrops-Usability-Round 2-KN-Complete.png IntelliCrops-Usability-Round 2-KN-Easy.png


Discussion
Issue Fix
Browser-specific display bug that prevented the content from showing up
  • Fixed; now cross-browser
Content displayed all bunched-up together
  • Added content detection to add breaks and paragraphing at suitable locations



Back to Home Page