IS480 Team wiki: 2016T2 ProgneSIS User Study

From IS480
Jump to navigation Jump to search
Team Prognesis Icon Home.png


  Team Prognesis Icon AboutUs.png


  Team Prognesis Icon Overview.png


  Team Prognesis Icon ProjectManagement.png


  Team Prognesis Icon Documentation.png


Technical Documents Design Documents Test Documents Meetings Minutes Project Deliverables

User Testing 1

  • Date: 12 October 2016
  • Venue: Respective offices of our potential users

User Study Design Document

User Study Objective

  • Find out if user interface is intuitive
  • Find out if the metrics are what they look at
  • Find out which metric(s) they look at more frequently
  • Identify usability problem(s) which users might face


  • 2 participants

These participants are actual users of VersaFleet(the application in which we build our application on). For confidentiality purposes, we would name them Tester 1 & 2.

Web Prototype

Below are views for the paper prototype used.

User Study Tasks

User Study task forms were given to the participants. These study tasks were designed to find out if users are able to navigate from one page to another without specific instructions as we want to find out if information display is clear and concise.

User Testing 1 Results

Study Goal Results Goal Achieved/Not
1. Users should find it easy to navigate from page to page when completing the tasks Expected average number of navigation errors was 5 while the real average number of navigation errors is 2.50. Goal Achieved
2. Users should find the system’s functions useful Expected average rating for the usefulness of systems features is 8, and the actual average rating of usefulness was 8. However, there is room for improvement in the qualitative aspect. Goal Achieved
3. Users should have clear understanding of the root cause of an alerted problem All participants filled in the right root cause of an alerted problem. Goal Achieved
4. Users should find complete task #3 and #7 without guidance from facilitator Although participants managed to complete task #3 and #7 within 1 minute, they sought clarifications with facilitators. Goal Unachieved

Team ProgneSIS US NE.png
Total No. of Navigation Errors Average No. of Navigation Errors Standard Deviation of Navigation Errors Median of Navigation Errors Expected No. of Navigation Errors
5 2.50 0.71 2.5 5

Team ProgneSIS US UFR.png

Average Usefulness of Function Rating Expected Usefulness of Function Rating
8 8

Team ProgneSIS US Task6.png

No. of Participants Answering Task 6 Correctly No. of Participants Answering Task 6 Incorrectly Expected No. of Participants Answering Task 6 Correctly
2 0 2

Team ProgneSIS US TTT3T7.png

Participant Time Taken to Complete Task 3 & 7
1 14
2 35

Average Time Taken to Complete Task 3 & 7 Standard Deviation of Time Taken Median of Time Taken Expected Average Time Taken
24.50 14.85 24.50 60

Likes Dislikes
Useful for evaluating business performance without the need for manual calculation Display of information should be more easily understandable
Overview of company's KPIs KPIs should be categorised
Delivery Fulfilment Metric is most frequently looked at when evaluating company's performance More details on operations would be good
Revenue and cost metrics may be helpful These metrics are not viewed frequently