HeaderSIS.jpg

IS480 Team wiki: 2010T1 The Green Reapers: Usability Testing

From IS480
Revision as of 15:46, 28 November 2010 by Edwin.feng.2008 (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Back to Home Page

To complement functional testing with our mentors at IRRI, we also pushed hard to improve the usability of our features. We reached out to untrained users to see how well they might take to the functions without any prompting.



Methodology

  • 2 rounds of usability testing
    • Iteration 6: 24 - 26 October
    • Iteration 7: 12 - 15 November
  • Task-based tests:
    • Unassisted as far as possible
    • Model real-life task goals
    • Focused on usability & discoverability
  • Mixed background of users
    • 75% non-technical
    • Mix of male and female, young and old




Test Questions

Analytics

IntelliCrops Feedback - Analytics
Please head over to http://www.intellicrops.com/analytics/ and create 3 different charts that show

1) the total number of calls that come from each country (pie) 2) the different average monthly call durations of each country (clustered columns) 3) the different average daily call durations of each country (clustered columns)

IntelliCrops-Usability-Analytics-Form.png
Question Type
Were you able to create the graphs successfully? Yes / No
How easy was it to create the graphs? Scale of (1) difficult - (7) easy
How easy was it to view the graphs? Scale of (1) bad - (7) good
Did you encounter any problems while using the IVR System? Textarea
Any other comments Textarea


Interactive Voice Response

IntelliCrops Feedback - Interactive Voice Response
(Requires a Skype account)

Please head over to http://www.intellicrops.com/wordpress/ and click on the Skype icon to start dialing

1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice

Press 0# to return to the main menu

2) Get a diagnosis for poor grain germination

IntelliCrops-Usability-IVR-Form.png
Question Type
Were you able to find the answers using the IVR system? Yes / No
Did you encounter any problems while using the IVR System? Textarea
How easy was it to use the IVR? Scale of (1) difficult - (7) easy
How was the quality of the voice? Scale of (1) bad - (7) good
Were the answers too lengthy? Yes / No
Any other comments Textarea


Knowledge Navigator

IntelliCrops Feedback - Knowledge Navigator
Please head over to http://www.intellicrops.com/ivr/knowledgenavigator.php

1) Find the factsheet on Organic Rice that tells us what to consider in producing organic rice

2) Get a diagnosis for poor grain germination

IntelliCrops-Usability-KN-Form.png
Question Type
Were you able to find the answers using the Knowledge Navigator? Yes / No
How easy was it to use the Knowledge Navigator? Scale of (1) difficult - (7) easy
Did you encounter any problems while using the Knowledge Navigator? Textarea
Any other comments Textarea




Results and Action Plan

Analytics

Usability Testing Round 1
IntelliCrops-Usability-Round 1-Analytics-Complete.png IntelliCrops-Usability-Round 1-Analytics-Easy.png IntelliCrops-Usability-Round 1-Analytics-View.png


Improved Results Observed During Usability Testing Round 2
IntelliCrops-Usability-Round 2-Analytics-Complete.png IntelliCrops-Usability-Round 2-Analytics-Easy.png IntelliCrops-Usability-Round 2-Analytics-View.png


Discussion
Issue Fix
Labels and values vague and ambiguous.
  • We previously assumed some familiarity with graphical analysis.
  • We've added hints and labelled previews to describe the nature and impact of the various fields




Interactive Voice Response

Usability Testing Round 2
IntelliCrops-Usability-Round 1-IVR-Completion.png IntelliCrops-Usability-Round 1-IVR-Easy.png IntelliCrops-Usability-Round 1-IVR-Quality.png


Improved Results Observed During Usability Testing Round 2
IntelliCrops-Usability-Round 2-IVR-Completion.png IntelliCrops-Usability-Round 2-IVR-Easy.png IntelliCrops-Usability-Round 2-IVR-Quality.png


Discussion
Issue Fix
Pressing the numbers didn’t seem to work
  • We assumed that most people have had experience with other interactive voice response systems, such that pressing hex after their option would be natural.
  • However, some participants neglected to do so, and were left wondering why the numbers didn’t work.
  • We solved this by adding more helpful user prompts. For example, we now ask user to press the number, and “hex”.
Poor help provided
  • When we interviewed the participants, we found that poor help referred to the inability to repeat options or return to the main menu. We added this functionality before the second round of tests.
Cannot really hear the menu options. Quite muffled
  • We investigated and found a few parameters to consider – network speed was certainly an issue, but it was out of our hands, so we focused on tweaking the speed of the synthesized voice, and added extra punctuation to accentuate the various points of emphasis.




Knowledge Navigator

Usability Testing Round 1
IntelliCrops-Usability-Round 1-KN-Complete.png IntelliCrops-Usability-Round 1-KN-Easy.png


Improved Results Observed During Usability Testing Round 2
IntelliCrops-Usability-Round 2-KN-Complete.png IntelliCrops-Usability-Round 2-KN-Easy.png


Discussion
Issue Fix
Browser-specific display bug that prevented the content from showing up
  • Fixed; now cross-browser
Content displayed all bunched-up together
  • Added content detection to add breaks and paragraphing at suitable locations



Back to Home Page