HeaderSIS.jpg

IS480 Team wiki: 2012T2 Team Prime Usability Tests

From IS480
Jump to navigation Jump to search

PrimeLogo.png

HOME PROJECT OVERVIEW PROJECT MANAGEMENT DOCUMENTATION

Latest Events
Notifications
Progress

 

Project Description
Stakeholders
The Team

 

Schedule & Scope
Milestones
Deliverables

 

Risks
Framework
Metrics

 

Diagrams
User Interface
User Tests

 

Meeting Minutes
Resources
Learning Outcomes


User Test 1

Objectives

OBJECTIVES:

  1. To obtain feedback from our users with regards to the features in the application
  2. To improve the usability of the application


Scope

The table below shows a list of features that were tested for our first user test. The features target parents (representing registered users of Swimix) and swimming instructors.

No. Features Reg. User Instructor
1 Register / Log in / Log out
2 Change Password
3 Update User Profile
N/A
4 Update Instructor Profile
N/A
5 Search for Class
N/A
6 Search for Instructor
N/A
7 Create and Remove Lesson Slot
N/A
8 Create and Remove Student Details
N/A

THE SESSION
User Test 1 was conducted successfully on 27 Jan 2013 at Yishun Swimming Complex.

  1. A total of 8 parents participated in the user test in the role of a Registered User.
  2. A total of 2 swimming instructors participated in the user test in the role of an Instructor.


Testing Methodology

Collecting of Qualitative Metrics

  1. Participants are encouraged to think aloud their thought process as they are performing each task. For example, we would encourage them to say whatever they are looking at, thinking, doing, and feeling as they go about the task. This enables us as the observer to better understand each participant’s thought processs and see first-hand the process of him/her completing the task.
  2. Facilitators will observe for usability issues during the testing procedure by recording down what participants say. The test sessions will also be video-recorded on Screen Flow so that we can go back and refer to what participants did and how they reacted.
  3. In addition, after participants have completed the user test, they will be asked to do a satisfaction level survey that will aid in the collection of qualitative metrics.

Collecting of Quantitative Metrics

  1. The amount of time taken to complete each task
  2. The number of clicks taken for each task


Registered Users


UserFeedback1.png

Solution: Place Login and Register in the same area and allow switching by tabs.

UserFeedback2.png

Solution: Change the View link to a button so users know that they can click on it to view the instructor’s profile


CONCLUSION
Base on the user feedbacks, we found out that users are mostly receptive to the idea of a swim related search portal. The top 2 favourite functions identified by the users were the Manage Search function and the Manage Notification function. They commented that the search function was easy to use and could be very useful to them. The notification function is also something they felt that is lacking right now which causes them to make wasted trip to the swimming pool only to find that the lesson was cancelled. The function that has the highest amount of users indicating that they are unlikely to use is the Instructor Review feature. They explained that they would not want to go through the trouble to register an account just to write a review for the instructors. However, they wouldn’t mind writing one if there is a simpler and more convenient alternative. One tester commented that he prefer to register with the instructor in person instead of looking at the instructor’s photo on the computer screen because by doing so, he could infer the instructor’s character and personalities. One of the possible solutions was to include a short introduction video clip of each instructor so that users are able to gauge on themselves through the video. In conclusion, many users expressed that they portal is user-friendly and would be very useful to them.



For a more detailed version of the user feedback and the team's response plan:

  1. Individual User Feedback (Registered Users)
  2. Response Plan to Feedback (Registered Users)


Instructors

InstructorFeedback1.png

Solution:
Use radio buttons instead of dropdown list.

InstructorFeedback2.png

Solution:
Display only the student name and contact number.
Instructors can choose to click on the student's name to view the rest of their information.


CONCLUSION
It was hard for us to get instructors to test our system for us because they is usually busy all the time while they are at the pool. They will either be teaching a class, or be on duty as a security guard. Thus, we only manage to get 2 instructors testers during their lunch break period when it is too hot to conduct a swim class. Both instructors found the portal user friendly and they did not face any problem using the portal. The provided us with mostly aesthetics feedback of the portal eg. Student list too cluttered, login button too small. One important point which they commented was that they are always on the go. Thus, they prefer to use the system on their smartphone rather than in front of the computer.


For a more detailed version of the instructor feedback and the proposed changes:

  1. Individual Instructor Feedback (Instructors)
  2. Response Plan to Feedback (Instructors)


UT1 Supporting Documents

Phase Supporting Documents
Before Testing
1. Execution Plan
During Testing
1. Scenario Task Lists
i) For Registered Users
ii) For Instructors
2. Qualtrics Surveys
i) For Registered Users
ii) For Instructors
3. Facilitator Script
After Testing
1. User Feedback
i) From Registered Users
ii) From Instructors
2. Qualtrics Survey Reports
i) Registered Users
ii) Instructors
3. Proposed Changes to Feedback
i) For Registered Users
ii) For Instructors

User Test 2

Objectives

OBJECTIVES:

  1. To obtain feedback from our users with regards to the features in our application so as to improve its usability
  2. To ensure that the features developed matches the expectations of the real users (parents & instructors)


Scope

The table below shows a list of features that were tested for our first user test. The features target parents (representing registered users of Swimix) and swimming instructors.

No. Features Reg. User Instructor
1 Register / Log in / Log out
2 Search for Class / Instructor
N/A
3 Pay for Class
N/A
4 Send Mass Notification to Students
N/A
5 Send Evaluation Survey to Students
N/A
6 Evaluate Instructor
N/A
7 Add New Lesson Slot
N/A
8 Add Student to Lesson Slot
N/A

Testing Methodology

Collecting of Qualitative Metrics

  1. Participants are encouraged to think aloud their thought process as they are performing each task. For example, we would encourage them to say whatever they are looking at, thinking, doing, and feeling as they go about the task. This enables us as the observer to better understand each participant’s thought processs and see first-hand the process of him/her completing the task.
  2. Facilitators will observe for usability issues during the testing procedure by recording down what participants say. The test sessions will also be video-recorded on Screen Flow so that we can go back and refer to what participants did and how they reacted.
  3. In addition, after participants have completed the user test, they will be asked to do a satisfaction level survey that will aid in the collection of qualitative metrics.

Collecting of Quantitative Metrics

  1. The amount of time taken to complete each task
  2. The number of clicks taken for each task


Registered Users

FEEDBACK:

Prime UT2Feedback2.png

Solution:
Replace “Please Select” with “No Preference”, and remove the unnecessary “Zone” search option.

Prime UT2Feedback3.png

Solution:
Replace “ Use a different color for the instructor’s name and bold it.

Prime UT2Feedback4.png

Solution:
Include a popup message to inform unregistered users to login first before making payment.

Instructors

FEEDBACK:
Prime UT2Feedback1.png

Solution:
Replace the photos with useful information on how the system works.

UT2 Supporting Documents

Phase Supporting Documents
Before Testing
1. Execution Plan
During Testing
1. Task Lists
i) For Registered Users
ii) For Instructors
2. Qualtrics Surveys
i) For Registered Users
ii) For Instructors
3. Facilitator Script
After Testing
1. User Feedback
i) From Registered Users
ii) From Instructors
2. Qualtrics Surveys
i) Registered Users
ii) Instructors
3. Proposed Changes to Feedback
i) For Registered Users
ii) For Instructors