HeaderSIS.jpg

IS480 Team wiki: 2017T1 TeamBFF UserTesting

From IS480
Jump to navigation Jump to search
TEamBFF-Banner.jpg


TeamBFFs-Home-Icon.png   HOME

 

TeamBFFs-AboutUs-Icon.png   ABOUT US

 

TeamBFFs-ProjectOverview-Icon.png   PROJECT OVERVIEW

 

TeamBFFs-ProjectManagement-Icon.png   PROJECT MANAGEMENT

 

TeamBFFs-Documentation-Icon.png   DOCUMENTATION



 

 




UAT Test Plan

Who

Malcolm (Lead Quality Assurance) leads the testing for every iteration

  1. Create an excel test plan for every iteration
  2. Conducts testing for every iteration
  3. Record all bugs and suggest bug cause (if possible) for every iteration
  4. Report to Eugene (Project Manager) at the end of every iteration

When

During every iteration, Malcolm (Lead QA) will have 2 window to conduct testing,

  1. Day 1 to 7 (Testing)
    1. To develop test cases
    2. To do testing on application
  1. Day 8 to 10 (Regression Testing)
    1. To test the entire application on server
    2. Testing will be based on all test cases done so far

Where

  1. Localhost: Through running on netbeans (QA to pull files from Git)
  2. Server: AWS Server. Group will deploy codes before day 8.

How

  1. Based on the bugs metrics, team will decide to fix bugs in this iteration or next.
  2. Project Manager can also call for a stop of development in order to clear bugs.


UAT 1


Test Plan

Venue: Singapore Weiqi Association (City), 116 Middle Road, ICB Enterprise House #03-04, 188972
Date: 19 July 2017, Wednesday
Time: 7.30pm
Duration: ~ 30 minutes per user
Number of Participants for Singapore Weiqi Association: 6
User Test: Instructions can be found here [1]
Participants roles: Players and Game Organizer

Objectives

  • To evaluate the system’s alignment with business requirement
  • To verify if system have met usability criteria through observing users’ behaviours
  • To further improve the UI/UX and overall system based on feedbacks

Scope of UAT 1

  • Registration
  • Login Auctions
  • Update Profile
  • Create Tournament
  • Register Player

Procedures

Each users are given test instruction to follow through the user testing and encouraged to think aloud. Team BFFs will guide users when they are unclear of the steps. Team BFFs member will be assigned to each user to record down the users’ behaviours. At the end of the user testing, team member will ask users about what they like and dislike about the user interface and a follow up feedback of the functions and execution of the function processes.

S/N Goals Reached?
1 Participants should be able to register a new account Goal almost reached. Participants were able to complete the tasks. One of the Participants took a longer time to register a new account.
2 Participants shoud be able to complete verification process of a new user easily with little or non-guidance [Function tested: Login] Goal reached. All participants completed the tasks.
3 Participants shoud be able to change their password easily with little or non-guidance [Function tested: Update Profile] Goal reached. All participants completed the tasks.
4 Participants shoud be able to register for the correct tournament easily with little or non-guidance Goal almost reached. Participants were able to complete the tasks. One of the Participants fail to register for the tournament.
5 Participants shoud be able to create a new tournament easily with little or non-guidance [Function tested: Create Tournament] Goal reached. All participants completed the tasks.

Key Findings

Functions Users' Comment Changes to be made
UI/UX Participants mentioned the error messages on login page was not clear enough Team will have to reflect and edit the error messages clearer
Function Refinement Participants mentioned that registration of user should requires PDPA agreement Team will have to include PDPA agreement with link checkbox
Function Refinement Participants mentioned that there is a need for PDF upload for more details on createtournament.jsp Team will have add in PDF upload.

Overall Results

  • Participants from Singapore Weiqi Association is satisfied with the progress


UAT 2


Test Plan

Venue: Singapore Weiqi Association (City), 116 Middle Road, ICB Enterprise House #03-04, 188972
Date: 16 Aug 2017, Wednesday
Time: 7.30pm
Duration: ~ 30 minutes per user
Number of Participants for Singapore Weiqi Association: 5
User Test: Instructions can be found here [2]
Participants roles: Players, Game Organizer and Judge

Objectives

  • To evaluate if the current workflow process of a tournament end-to-end and the system is aligning with business requirement
  • To determine the usability of Game Organising module in specifics to the pairing and judging functions through observing users’ behaviours
  • To further improve the UI/UX and overall system based on feedbacks

Scope of UAT 2

  • Tournament with criteria locking
  • Registration of new user
  • Tournament registration
  • Pairing a single elimination tournament
  • Judging a tournament

Procedures

Each users are given test instruction to follow through the user testing and encouraged to think aloud. Team BFFs will guide users when they are unclear of the steps. Team BFFs member will be assigned to each user to record down the users’ behaviours. At the end of the user testing, team member will ask users about what they like and dislike about the user interface and a follow up feedback of the functions and execution of the function processes.

S/N Goals Reached?
1 Participants should be able to create a tournament with little or no guidance Goal reached. All participants was able to complete the tasks.
2 Participants should be able to assign judge with little or no guidance Goal reached. All participants was able to complete the tasks
3 Participants should be able to complete verification process of a new account with little or no guidance Goal reached. All participants was able to complete the tasks
4 Participants should be able to register for a tournament that is available with little or no guidance Goal almost reached. All participants was able to complete the tasks. One participant took a longer duration to navigate the page but managed to complete tasks with assistance
5 Participants should be familiar with the tournament management with little or no guidance (Function tested: Generate pairing, Close registration) Goal reached. All participants was able to complete the tasks
6 Participants should be able to view his/her pairing (Function tested: View pairing) Goal reached. All participants was able to complete the tasks
7 Participants should be able to judge tournament with little or no guidance Goal almost reached. All participants was able to complete the tasks. Two participants took a slightly longer time to reach the judge page. One participant faced bugs when judging a match.
8 Participants should be able to close round when a tournament round ended with little or no guidance Goal reached. All participants was able to complete the tasks

Key Findings

Functions Users' Comment Changes to be made
UI/UX Participants mentioned that for judge.jsp absent function and select winner function should on different view to avoid confusion Team will create 2 different buttons and popup screens for absent and select winner function in judge.jsp
Function Refinement Participants mentioned some of the chinese players would prefer use their chinese names as their account names Team will have to optional chinese name in create user functionality

Overall Results

  • Clients were pretty satisfied with the progress. Clients mentioned the User Interface can be further improved


UAT 3


Test Plan

Venue: Singapore Weiqi Association (City), 116 Middle Road, ICB Enterprise House #03-04, 188972
Date: 23 Sep 2017, Monday
Time: 7.30pm
Duration: ~ 30 minutes per user
Number of Participants for Singapore Weiqi Association: 5
User Test: Instructions can be found here [3]
Participants roles: Players, Game Organizer and Judge

Objectives

  • To determine the usability of Game Organizing and Result Display modules in specifics to the pairing and tournament results through observing users’ behaviours
  • To ascertain the effectiveness of our system pairing logic and its consistency to OpenGotha
  • To further improve the UI/UX and overall system based on feedbacks

Scope of UAT 3

  • Account module (Registration, Login)
  • Create tournament
  • Tournament registration
  • Judging a tournament
  • Result Display(Tournament standings, Round result)

Procedures

Each users are given test instruction to follow through the user testing and encouraged to think aloud. Team BFFs will guide users when they are unclear of the steps. Team BFFs member will be assigned to each user to record down the users’ behaviours. At the end of the user testing, team member will ask users about what they like and dislike about the user interface and a follow up feedback of the functions and execution of the function processes.

S/N Goals Reached?
1 Participants should be able to register a new account Goal reached. Participants was able to complete the tasks.
2 Participants should be able to create a tournament with little or no guidance Goal reached. All participants was able to complete the tasks
3 Participants should be able to assign judge with little or no guidance Goal reached. All participants was able to complete the tasks
4 Participants should be able to complete verification process of a new account with little or no guidance Goal reached. All participants was able to complete the tasks
5 Participants should be able to register for a tournament that is available with little or no guidance Goal reached. All participants was able to complete the tasks.
6 Participants should be familiar with the tournament management with little or no guidance (Function tested: Generate pairing, Close registration) Goal almost reached. All participants was able to complete the tasks. One participant took a longer duration to navigate the page but managed to complete tasks with assistance
7 Participants should be able to view his/her pairing (Function tested: View pairing) Goal reached. All participants was able to complete the tasks
8 Participants should be able to judge tournament with little or no guidance Goal almost reached. All participants was able to complete the tasks. One participants took a slightly longer time to reach the judge page. One participant faced bugs when judging a match.
9 Participants should be able to close round when a tournament round ended with little or no guidance Goal reached. All participants was able to complete the tasks
10 Participants should be able to view past round result and standings when a tournament round ended with little or no guidance Goal reached. All participants was able to complete the tasks

Key Findings

Functions Users' Comment Changes to be made
UI/UX Participants mentioned that, as a game organizer, it is difficult to know what is the status of the tournament (e.g Pairing Management, Setup) Team will add status of the tournament on every screens of tournament management pages.

Overall Results

  • Clients were extremely satisfied with the progress and the new User Interface.