HeaderSIS.jpg

IS480 Team wiki: 2018T1 Tempest User Testing

From IS480
Jump to navigation Jump to search

FYP Banner (with lightning, with background).png

HOME

   

ABOUT US

   

PROJECT OVERVIEW

   

PROJECT MANAGEMENT

   

DOCUMENTATION

Personas Diagrams Prototype Technology User Testing Minutes Deliverables


UAT 1 Details - Bootstrap Platform

Venue: Monsoon@Novena
Date: 28 June 2018
Time: 7pm to 10pm
Number of Participants:

  • 1 Outlet Receptionist
  • 2 Outlet Staff (1 playing the role as customer)

Objectives

  • Gather user feedback on developed user interface for developed functions from real users
  • Identify potential usability issues

Scope

Customer

  • Create a customer account
  • Login with newly created account
  • Change password
  • Book appointment
  • View upcoming appointment(s)
  • View appointment history

Admin

  • Add a new product
  • View all products
  • Edit product details
  • Delete product
  • View current appointments
  • Mark appointments as complete
  • Add stylists' availablity
  • Delete stylists' availability

Procedure

We have assigned roles to the team members, which includes:

  • A main facilitator, who will facilitate the test with a fixed set of instructions
  • A secondary facilitator, to collect feedback at the end of the user testing session
  • A scribe, who will be recording behaviors and comments from our participants if participants think aloud while testing
S/N Objective Attained?
1 Participant should be able to create a customer account Passed
2 Participant should be able to login with created account Passed
3 Participant should be able to change password Passed with some assistance
4 Participant should be able to book appointment Passed
5 Participant should be able to view upcoming appointment(s) Passed
6 Participants should be able to view appointment history Passed


S/N Objective Attained?
1 Participant should be able to add a new product Passed
2 Participant should be able to view all products Passed
3 Participant should be able to edit product details Passed
4 Participant should be able to delete product Passed
5 Participant should be able to view current appointments Passed
6 Participants should be able to mark appointments as complete Passed
7 Participants should be able to add stylists' availablity Passed
8 Participants should be able to delete stylists' availability Passed


Key Findings

Type Functions Feedback Changes
UI Appointment Booking Show Calendar View instead of Drop-down List of dates UI will incorporate feedback
UI Appointment Booking Show confirmation message after booking appointment UI will incorporate feedback
UI Appointment Booking Stylists should be able to view customer details from view current appointments page Coders will incorporate feedback
UI Inventory Page UI should be more beautiful rather than a plain table showing all products UI will incorporate feedback
UI Inventory Page UI should be more seamless by reducing the number of fields to fill in UI will incorporate feedback
UI Change Password Location of Change Password function is not intuitive UI will move it from login page, to menu of functions for logged-in users

UAT 2 Details - After migration to Wordpress

Venue: Monsoon@Novena
Date: 31 July 2018
Time: 7pm to 10pm
Number of Participants:

  • 1 Outlet Receptionist
  • 2 Outlet Staff (1 playing the role as customer)

Objectives

  • Gather user feedback on developed user interface for developed functions from real users
  • Identify potential usability issues

Scope

Customer

Authentication

  • Create a customer account
  • Login with newly created account
  • Change password

Appointment Management

  • Book appointment
  • Confirm appointment
  • Cancel appointment

E-store

  • View product
  • Add product to cart
  • Remove product from cart
  • Checkout and make payment

Live-chat

  • Start live chat

Translation

  • Translate page to chinese


Admin

E-store

  • Add a new product
  • View all products
  • Edit product details
  • Delete product

Admin Appointments Dashboard

  • View current appointments
  • Add stylists' availablity
  • Delete stylists' availability
  • View history by product sales and stylists sales

Live-chat

  • Login/Logout of live chat
  • Reply to messages on live chat

Procedure

We have assigned roles to the team members, which includes:

  • A main facilitator, who will facilitate the test with a fixed set of instructions
  • A secondary facilitator, to collect feedback at the end of the user testing session
  • A scribe, who will be recording behaviors and comments from our participants if participants think aloud while testing
S/N Objective Attained?
1 Participant should be able to create a customer account Passed
2 Participant should be able to login with created account Passed
3 Participant should be able to change password Passed with function bug noted
4 Participant should be able to book appointment Passed
5 Participants should be able to confirm appointment Passed
6 Participants should be able to cancel appointment Passed
7 Participants should be able to view e-store products Passed
8 Participants should be able to add product to cart Passed
9 Participants should be able to remove product from cart Passed
10 Participants should be able to checkout and make payment Passed
11 Participants should be able to start live chat and receive admin's replies Passed
12 Participants should be able to translate page to chinese Passed with function bug noted


S/N Objective Attained?
1 Participant should be able to add a new product Passed
2 Participant should be able to view all products Passed
3 Participant should be able to edit product details Passed
4 Participant should be able to delete product Passed
5 Participant should be able to View current appointments Passed
6 Participants should be able to add stylists' availablity Passed
7 Participants should be able to delete stylists' availability Passed
8 Participants should be able to view history by product sales and stylists sales Passed
9 Participants should be able to login/Logout of live chat Passed
10 Participants should be able to reply to messages on live chat Passed


Key Findings

Type Functions Feedback Changes
Function Translation Website switched to Chinese without user requesting for it Follow up in progress
Function and UI New member registration New member registration confirmation should have sent change password link, but it didn’t work. Sponsor also indicated that we shouldn’t even need this confirmation, set password during registration itself UI will incorporate feedback
Function Appointment Booking Appointment booking exporting to excel had some data wrongly populated Follow up in progress
UI E-store Various UI improvement points noted.
  • Remove the blue line beside images
  • Resize images
  • Set font for product description to fit within 3 lines
  • Put more images under the slider for related product types
  • Neaten the alignment for images in homepage
UI will incorporate feedback
UI Mobile view Alignment is off on both tablet and phone. UI will incorporate feedback

UAT 3 Details - Before Mid-Terms

Venue: SOB SR2.2 and Online
Date: 27 Sep 2018
Time: 12 noon - 5pm
Number of Participants:

  • 53 Customers (38 Potential Customers as represented by students, 15 Existing Customers)
  • 4 Outlet Receptionist/Stylists
  • 3 HQ Staff

Objectives

  • Assess user friendliness for developed functions
  • Identify usability issues
  • Gather more discretionary feedback on website in general

Scope

Customer

Authentication

  • Create a customer account
  • Login with newly created account

General site navigation

  • Find treatment (service)

Appointment Management

  • Book appointment

E-store

  • Find a product with certain attributes

Live-chat

  • Try live chat

All other functions (e.g. translation, media)

  • Free-play for user to provide feedback

Outlet Receptionist/Stylist

Admin Appointments Dashboard

  • Manually add appointment
  • View appointments

HQ Staff

E-store

  • Add new product
  • Edit inventory quantity of existing product
  • Add e-store coupon code

Dashboard

  • Add new branch outlet
  • Generate appointment reports
  • Obtain specific information from reports generated

Procedure

We have assigned roles to the team members, which includes:

  • A main facilitator, who will facilitate the test with a fixed set of instructions
  • A secondary facilitator, to assist users with any questions they may have during the test
  • A scribe, who will be recording behaviors and comments from our participants if participants think aloud while testing
S/N Objective Attained?
1 Participant should be able to create a customer account Passed
2 Participant should be able to login with created account Passed
3 Participant should be able to find a certain treatment (service) Passed with assistance, as participants did not know the treatment was a service and not a product. The flaw was in our test instruction, which we quickly rectified.
4 Participant should be able to book appointment Passed
5 Participants should be able to view e-store products Passed
6 Participants should be able to use filters to find products with certain attributes Passed
7 Participants should be able to try the live chat Passed


S/N Objective Attained?
1 Participant should be able to manually add appointment for customers who call in Passed
2 Participant should be able to view appointments Passed
S/N Objective Attained?
1 Participant should be able to add new product in the e-store Passed
2 Participant should be able to edit inventory quantity of existing product Passed
3 Participant should be able to add e-store coupon code Passed


4 Participant should be able to add new branch outlet Passed
5 Participant should be able to generate appointment reports Passed
4 Participant should be able to obtain specific information from reports generated Passed

Key Findings

Type Functions Feedback Changes
Performance Issue Website in general Some customers found the website laggy. This laggy performance was not widespread, but it was reported by a sizeable minority of users (around 8 users out of 53) Follow up in progress
UI Website in general Customers would like to see more photos of actual customers getting their hair done UI will liaise with sponsor and incorporate feedback
Function Search function in e-store Search button function can be further improved to search the entire site Follow up in progress
Function Suggestion for new function Customers wanted to find more reviews on the website. They suggested having a stylist review on stylist page Follow up in progress
Function Suggestion for new function Email reminder when appointment date is nearing Follow up will be done during recess week

UAT 4 Details - Focus on customer-facing functions

Venue: SIS SR 3-4 and Online
Date: 26 - 30 October 2018
Time: 12 noon - 5pm
Number of Participants:

  • 39 Customers

Objectives

  • Assess improvements made from mid-terms, for customer-facing functions
  • Identify usability issues

Scope

Customer

Authentication

  • Create a customer account
  • Login with newly created account

General site navigation

  • Use search function to find product

Appointment Management

  • Book appointment
  • Cancel appointment from their account

Quiz

  • Try quiz to get recommended next treatment

Procedure

We have assigned roles to the team members, which includes:

  • A main facilitator, who will facilitate the test with a fixed set of instructions
  • A secondary facilitator, to assist users with any questions they may have during the test
  • A scribe, who will be recording behaviors and comments from our participants if participants think aloud while testing
S/N Objective Attained?
1 Participant should be able to create a customer account Passed
2 Participant should be able to login with created account Passed
3 Participant should be able to use search function to find product Passed
4 Participant should be able to book appointment Passed
5 Participants should be able to cancel appointment from their account Passed
6 Participants should be able to do the quiz to get a recommendation on their next treatment Passed

Key Findings

Type Functions Feedback Changes
Screen Resolution Issue Website in general A sizeable number of customers found the website unresponsive to their particular screen resolution Resolved
Performance Issue Website in general Some customers found the website laggy. The number decreased since last UT, but it persists for some users Follow up in progress
Function Search function in e-store Not all users were able to locate the search function Follow up in progress

UAT 5 Details - Focus on admin-side functions

Venue: Monsoon HQ Office and Salon
Date: 8 November 2018
Time: 3 - 5pm
Number of Participants:

  • 3 HQ staff
  • 6 Stylists

Objectives

  • Assess usability of dashboards for HQ staff and stylists
  • Gather feedback on how valuable existing dashboard is and how to improve value further

Scope

HQ staff and Stylists

HQ Dashboard features

  • View dashboard for sales, number of customers, number of appointments, popular products for last 7 days
  • View same dashboards for the month
  • View dashboard showing past data for Bukit Timah, filtering by specific date or across month

Stylist Dashboard feature

  • View appointments for today
  • View appointments for next week

Procedure

We have assigned roles to the team members, which includes:

  • A main facilitator, who will facilitate the test with a fixed set of instructions
  • A secondary facilitator, to assist users with any questions they may have during the test
  • A scribe, who will be recording behaviors and comments from our participants if participants think aloud while testing
S/N Objective Attained?
1 HQ Participant should be able to easily view dashboard for sales, number of customers, number of appointments, popular products for last 7 days (easily, as defined by rating 4 and above on 5-point rating scale) Passed
2 HQ Participant should be able to easily view same dashboards for the month (easily, as defined by rating 4 and above on 5-point rating scale) Passed
3 HQ Participant should be able to easily view dashboard showing past data for Bukit Timah, filtering by specific date or across month (easily, as defined by rating 4 and above on 5-point rating scale) Passed
4 Stylist Participant should be able to easily view appointments for today (easily, as defined by rating 4 and above on 5-point rating scale) Passed
5 Stylist Participant should be able to easily view appointments for next week (easily, as defined by rating 4 and above on 5-point rating scale) Passed

Key Findings

Type Functions Feedback Changes
General Dashboard features All participants remarked on the ease of usage and value of the dashboards. None from UT, but we will continue enhancing the look of the dashboards