HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2013T1 ThunderBolt Project Documentation Usability Test"

From IS480
Jump to navigation Jump to search
Line 425: Line 425:
 
|}
 
|}
 
<br>
 
<br>
User Testing 2 has been expanded from User Testing 1 to include 5 more features, which are highlighted in yellow.
+
'''User Testing 2 has been expanded from User Testing 1 to include 5 more features, which are highlighted in yellow.'''
  
 
====<font color="#200772">Testing Method</font>====
 
====<font color="#200772">Testing Method</font>====

Revision as of 15:16, 17 October 2013

Wikiprofilepic.jpg

Home

  About Us   Project Overview   Project Management   Project Documentation
Diagrams Prototype Usability Test Meeting Minutes Downloads

Usability Test 1

Date and Venue

Date: 2 Sep 2013
Time: 1.30pm - 3pm
Venue: SIS MR4.7

Participants

No. of Participants: 10
Participant Profile:
- Admin
- Course Coordinator
- Supervisor/Reviewer
- Students doing FYP next semester

UT1Photo.png
UT1Photo2.png
UT1Photo3.png

Scope of Test

Function Admin Course Coordinator Supervisor/Reviewer Student
Single Sign On Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Role switching - Transparent-green-checkmark.png - -
Create schedule Transparent-green-checkmark.png Transparent-green-checkmark.png - -
View schedule Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Create booking Transparent-green-checkmark.png Transparent-green-checkmark.png - Transparent-green-checkmark.png
View booking Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Edit booking Transparent-green-checkmark.png Transparent-green-checkmark.png - -
Delete booking Transparent-green-checkmark.png Transparent-green-checkmark.png - Transparent-green-checkmark.png
Approve/Reject booking - - Transparent-green-checkmark.png -
View booking history Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Supervisor Availability - - Transparent-green-checkmark.png -
Add default milestones Transparent-green-checkmark.png Transparent-green-checkmark.png - -
Add required attendees for different milestones Transparent-green-checkmark.png Transparent-green-checkmark.png - -
Set presentation duration Transparent-green-checkmark.png Transparent-green-checkmark.png - -

Testing Method

A list of tasks based on different scenarios were given to participants and they need to follow the scenarios and figure out how to complete them in the system. Respective facilitators who sat beside them need to observe and record down how they were using the system to complete the task assigned.
Upon completion of all tasks, participants were given a link to complete a questionnaire with regards to their experiences of using our system and feedback any issues they faced during testing or improvements they would like to see in the future.

Test Results

When asked about what do they like about our system:
- 9 out of 10 said that UI is clean and intuitive
- 1 out of 10 said that new system, better than nothing

What asked what they dislike about the system:
- 2 out of 10 said that they are not sure how to check profile information
- 1 out of 10 said that system is slow
- 1 out of 10 not sure they can check booking history
- 1 out of 10 said that no confirmation when delete booking
- 1 out of 10 said that create schedule for Admin and Course Coordinator is not intuitive
- 1 out of 10 said that abit too many clicks
- 1 out of 10 encountered errors while performing tasks

Some of the issues they faced when performing tasks are:
- 2 out of 10 said that they have problem checking their role in the system
- 1 out of 10 face DB connection error
- 1 out of 10 said that Safari browser unable to click down arrow
- 1 out of 10 said that date and time field for edit booking is hard to use
- 1 out of 10 said that they cant use SSO auto login, need to have manual login
- 1 out of 10 said that email content is not sufficient

Improvements they would like to have to have a better user experience:
- 1 out of 10 wants the system to be faster
- 1 out of 10 want to improve on buttons e.g Booking history
- 1 out of 10 wants email header to be color-coded, easier for user to identify the status of the booking
- 1 out of 10 wants lesser clicks
- '1 out of 10 wants tool-tip/Breadcrumbs

Click here to see our Response Plan: [1]

SurveyResult1.PNG
SurveyResult2.PNG

SurveyResult3.PNG
SurveyResult4.PNG

Most Common Feedback for User Testing 1:
UserTesting1MostCommonFeedback.PNGUserTesting1MostCommonFeedback 2.PNG

UserTesting1MostCommonFeedback 3.PNGUserTesting1MostCommonFeedback 3 fixed.PNG

Documentation

1. Test Plan
  1. Test plan: [2]
2. Scenario Task Lists
  1. Admin task: [3]
  2. Course Coordinator task: [4]
  3. Supervisor/Reviewer task: [5]
  4. Student task: [6]
3. Qualtrics Survey Questions
  1. Survey Link: [7]
  2. Survey Raw Result:[8]
4. Participant Informed Consent Form
  1. Consent Form: [9]

Usability Test 2

Date and Venue

Date: 4 Oct 2013
Time: 3.30pm - 7pm
Venue: SIS Seminar Rm 2-2

Participants

No. of Participants: 21
Participant Profile:
- Admin
- Course Coordinator
- 3 Supervisor/Reviewer
- 2 TA
- 15 Students completed IS480 and students doing IS480 next term
UT2Photo.pngUT1VSUT2participantprofile.png

Scope of Test

Function Admin Course Coordinator Supervisor/Reviewer TA Student
Single Sign On Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Role switching - Transparent-green-checkmark.png - - -
Create schedule Transparent-green-checkmark.png Transparent-green-checkmark.png - - -
View schedule Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Edit schedule Transparent-green-checkmark.png Transparent-green-checkmark.png - - -
Archive schedule Transparent-green-checkmark.png Transparent-green-checkmark.png - - -
Create booking Transparent-green-checkmark.png Transparent-green-checkmark.png - - Transparent-green-checkmark.png
Add optional attendees Transparent-green-checkmark.png Transparent-green-checkmark.png - - Transparent-green-checkmark.png
View booking Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Edit booking Transparent-green-checkmark.png Transparent-green-checkmark.png - - -
Delete booking Transparent-green-checkmark.png Transparent-green-checkmark.png - - Transparent-green-checkmark.png
Approve/Reject booking - - Transparent-green-checkmark.png - -
Reject booking with reason - - Transparent-green-checkmark.png - -
View my bookings Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png Transparent-green-checkmark.png
Supervisor Availability - - Transparent-green-checkmark.png - -
TA sign up for filming - - - Transparent-green-checkmark.png -
Add default milestones Transparent-green-checkmark.png Transparent-green-checkmark.png - - -
Add required attendees for different milestones Transparent-green-checkmark.png Transparent-green-checkmark.png - - -
Set presentation duration Transparent-green-checkmark.png Transparent-green-checkmark.png - - -


User Testing 2 has been expanded from User Testing 1 to include 5 more features, which are highlighted in yellow.

Testing Method

A list of tasks based on different scenarios were given to participants and they need to follow the scenarios and figure out how to complete them in the system. Respective facilitators who sat beside them need to observe and record down how they were using the system to complete the task assigned.
Upon completion of all tasks, participants were given a link to complete a questionnaire with regards to their experiences of using our system and feedback any issues they faced during testing or improvements they would like to see in the future.

Test Results

UT2SurveyResult1.pngUT2SurveyResult2.png

UT2SurveyResult3.pngUT2SurveyResult4.png

Most common feedback from User Testing 2
- Admin and Course Coordinator want to allow drag-n-drop when editing a booking
- Admin and Course Coordinator also want to request for reason when students delete booking
- Supervisors want their time slots to be pre-populate as unavailable instead of available
- Students are not sure if their supervisor/reviewers receive email to approve their booking
- TA wants to see other TAs' sign on on their availability page as well, so that they can swap with each other if can't make it last min
Click here to see our Response Plan: [10]
UserTesting2MostCommonFeedback.pngUserTesting2MostCommonFeedback 2.png

UserTesting2MostCommonFeedback 3.pngUserTesting2MostCommonFeedback 4.png

UserTesting2MostCommonFeedback 5.pngUserTesting2MostCommonFeedback 6.png

Documentation

1. Test Plan
  1. Test plan: [11]
2. Scenario Task Lists
  1. Admin task: [12]
  2. Course Coordinator task: [13]
  3. Supervisor/Reviewer task: [14]
  4. TA task: [15]
  5. Student task: [16]
3. Qualtrics Survey Questions
  1. Survey Link: [17]
  2. Survey Raw Result:[18]
4. Participant Informed Consent Form
  1. Consent Form: [19]