HeaderSIS.jpg

IS480 Team wiki: 2014T1 One Piece Project Testing

From IS480
Jump to navigation Jump to search

OnePiece Banner.jpg

Home Team One Piece Project Overview Project Management Documentation
Diagrams Prototype Testing Technology Meeting Minutes Archive Documents Current Product

UAT 1: Heuristic Evaluation

Objective

  • To evaluate usability of our first prototype (Low-Fi) using heuristic evaluation method

Scope

 Login Page
 Trainer/Trainee Homepages
 Create Module Page
 Create Quiz Page
 Take Module Page
 Take Quiz Page
 Performance Statistic Page

Background

 Four SMU SIS students
 IDP classmates
 1 Male & 3 Female

Collated Feedback after UAT & Changes

View our Compiled Heuristic Evaluation Feedback here

  • Problem – Trainer homepage data table columns are difficult to understand and too clustered.
  • Suggestion – Use icons instead of word descriptions.

UAT1 Change1 OnePiece.jpg


  • Problem – The link to direct back to login page is missing in most of the web pages.
  • Suggestion – Include standard navigation bar across all pages.

UAT1 Change2 OnePiece.jpg


  • Problem – Lack of detailed information (e.g. attempts) in viewing quiz results.
  • Suggestion – Add required information as new columns.

UAT 2: Hi Fi Prototype Testing

Objective

  • To get feedback on layout and usability of our improved prototype (Hi -Fi)

Scope

 Create Module Page
 Create Quiz Page
 Take Module Page
 Take Quiz Page
 Performance Statistic Page

Background

 Three SMU friends
 2 Male & 1 Female
 Non-IS students

Study Data & Results

Study Design
Study Task
Study Questionnaire
Study Data Sheet
Compiled Study Result

Qualitative Data

UAT2 Feedback1.JPG

Quantitative Data

UAT2 Feedback Trainer.JPG


UAT2 Feedback Trainee.JPG

Feedback (Problems)

 Difficult to navigate around and lack of instruction.
 Navigation bar need to be improved. Doesn't have a clue of which page I am at.
 Inconsistency in button colors based on different functions.

Main Changes

Change 1

 Standardized button theme.

UAT2 Change1 OnePiece.jpg

Change 2

Add more fields in creating/editing quiz

UAT2 Change2 OnePiece.jpg

Change 3

 Change trainer publish/recall function status

UAT2 Change3 OnePiece.jpg

Change 4

Add current page name across all sites

UAT 3: User Validation

Objective

  • To validate user requirement before acceptance and functionality testing

Scope

 Login Page
 Trainer Homepages
 Create Module Page
 Create Quiz Page

Background

 Two Clients (Mr. Tan and Mr. Morrison)
 Stakeholders of the BP Academy project

Suggestions for Improvement

  • Accepted the current functionality and workflow
  • Requested to add weight-age for each question as a new requirement
  • Requested to implement file upload for multiple users in Super Admin module
  • Requested to make the changes for user authentication module (Login) with WinNetwork ID.

UAT 4: System Usability


Picture UAT4.JPG


Objective

  • To display all developed functionality to the clients and get their perspective on the features, layout and usability

Scope

 Login Page
 Trainer/Trainee Homepages
 Create Module Page
 Create Quiz Page
 Take Module Page
 Take Quiz Page
 CRUD of user/user groups
 Upload mulitple user/user groups

Background

  • 6 Users from BP Singapore from Trading, Infrastructure and Operations units

Results

UAT4 Result1.JPG UAT4 Result2.JPG

Feedback

  • Fonts too small
  • Do not understand CRUD icon should provide more tool tips for users (especially in text editor)
  • The icons in Trainer Home Page are not visible and the labels are not clear



UAT 5: Split A/B Tesing


UAT5 execution.jpg


Objective

  • Split A/B Testing to see if the timer and checkpoint features enhances learning

Scope

 Login Page
 Trainer/Trainee Homepages
 Create Module Page
 Create Quiz Page
 Take Module Page
 Take Quiz Page
 CRUD of user/user groups
 Upload mulitple user/user groups

Background

  • 30 SMU students

UAT5 Execution.JPG


A/B Testing

ABTesting OnePiece.JPG ABTesting2 OnePiece.JPG


Feedback

UAT5 Result1.JPG UAT5 Result2.JPG


Problems

  • No indication that clicks are not usable in viewing the slides.
  • Tooltips for the text editor/homepage data table is needed.
  • In creating quiz, the fields such as passing marks are not visible and difficult to access after creating many questions.


Analysis: UAT 4 & 5

Feedback Comparison


UAT5Vs6 1.JPG UAT5Vs6 2.JPG


Changes for Improving Usability


More visible icons and clearer command description

UAT56 Change1.JPG



Easily visible & accessible passing marks fields after creating many questions.

UAT56 Change2.JPG



Clearer command in Viewing Slides

UAT56 Change3.JPG


UAT 6: Unit Testing


UAT-6 Execution


Objective

  • To display the newly developed functionalities to the clients and get their perspective on the features, layout and usability

Scope

 Create Module with Text Editor (Trainer)
 Performance Statistics (Trainer)
 Reporting (Trainer)
 Performance Statistics (Trainee)

Demographic

  • 5 Users from BP Singapore from Trading, Infrastructure and Operations units

Documents

OnePiece Task Instruction - UAT6

Results

UAT-6-results

Qualitative Feedback

  • Very intuitive performance statistics for Trainer and Trainee
  • Cannot change font for specific text in Text Editor
  • Fonts are too small
  • In viewing defaulting employees, the window should be bigger
  • Not too sure what are the icons in text editor
  • In Trainer performance statistics, help button is not very visible



UAT 7: Split A/B/C/D Testing


UAT7 execution1.jpg

UAT7 execution2.jpg


Objective

  • To see the effectiveness of the interactive learning features such as timer and pop quiz

Scope

 Take Module (Trainee)
 Take Quiz (Trainee)

Demographic

  • 40 SMU students from different faculties
UAT-7 Setting

Documents

OnePiece Task Instruction - UAT7

Results

Conclusion: Group D with Timer and Pop Quiz performs the best in the quiz, and thus the interactive learning features truly give benefits to the trainees.


UAT7 statistics analysis.JPG





UAT 8: End-to-end System Testing


UAT8 execution.jpg


Objective

  • To display all developed functionalities to the clients and get their perspective on the features, layout and usability

Scope

 CRUD of User Groups (Super-Admin)
 CRUD of Users (Super-Admin)
 Create Module with Text Editor (Trainer)
 Create Quiz (Trainer)
 Performance Statistics (Trainer)
 Take Module and Quiz (Trainee)
 Performance Statistics (Trainee)

Demographic

  • 2 BP Sponsors

Documents

OnePiece Task Instruction - UAT8

Results

UAT-6-results

Qualitative Feedback

  • Super Admin is easy to use
  • Performance Statistics is impressive
  • Need confirmation alerts before deleting the slides
  • Error in showing the answer choices in take quiz




Improvements after UATs


Clearer labels and help assistance for Text Editor
Change1before.JPG

Change1after.JPG



More comprehensive icons and visible labels
Change2before.JPG

Change2after.JPG