HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2013T2 Zora Testing"

From IS480
Jump to navigation Jump to search
Line 1,298: Line 1,298:
  
 
|-
 
|-
! colspan="6" style="background: #A9F5E1;" | General Data
+
! colspan="5" style="background: #A9F5E1;" | General Data
 
|-
 
|-
  
Line 1,324: Line 1,324:
 
|-
 
|-
 
|Browser
 
|Browser
|Google Chrome
 
 
|Google Chrome
 
|Google Chrome
 
|Google Chrome
 
|Google Chrome

Revision as of 22:56, 11 April 2014

Zora page header.png
Home Project Overview Project Management Documentation

Test Methodology

Zora test methodology.png

Zora Testing6.JPG

Test Plan

Zora test plan.png

Test Cases

Zora Testing14.JPG
Click here to view Login/Logout Test Case file.
Click here to view Remuneration Test Case file.
Click here to view Claim Test Case file.
Click here to view Leave Test Case file.
Click here to view Training & Development Test Case file.
Click here to view Performance Management & Appraisal Test Case file.
Click here to view Administration Test Case file.

Test Outcome

Heuristic Evaluation 1

Testers are the expert evaluators. They used the following heuristic and severity ratings, as shown below, to evaluate our application

Heuristics Severity Ratings

1. Visibility of system status

2. Match between system and the real world

3. User control and freedom

4. Consistency and standards

5. Error prevention

6. Recognition rather than recall

7. Flexibility and efficiency of use

8. Aesthetic and minimalist design

9. Help users recognize, diagnose, and recover from errors

10. Help and documentation

Not related to any heuristic

0 = I don't agree that this is a usability problem

1 = Cosmetic problem only

2 = Minor usability problem

3 = Major usability problem

4 = Usability catastrophe

Images taken during the evaluation

Yun Xi bringing Hui Shia through the remuneration part of the project
Mei Zhen explaining our prototype to Jing Yi
Yun Xi going through the prototype with Eng Sen
Yi Xian going through our prototype to Ken Khoon

Results obtained from the evaluation conducted

Zora Testing20.JPG






















User Test 1

Steps taken to conduct user testing 1
Post-Test
Identify:

  1. Goals of the test
  2. Scope of the test
  3. Participant Profile

Decides on:

  1. Testing set up (venue, time etc.)

Prepare:

  1. Testing items and documents

Recruit Testers

  1. Testers to read the study script


During-Test

  1. Give user informed form to testers
  2. Give task instructions to testers
  3. Give interview form to testers
  4. Observe and fill up Observation Form
  5. Give Questionnaire to testers to fill up while testing


Post-Test

  1. Consolidate Feedbacks
  2. Analyze Results
  3. Maintain a Response Plan
  4. Implement feedback in UT1 and test again in upcoming UT2


Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Pilot Participant Participant 1 Participant 2 Participant 3
IMG 4600.JPG
Zora Testing22.JPG
Zora Testing23.JPG
Zora Testing24.JPG
General Data
Age 35 50 23 26
Occupation Business Development Manager Administrator Student Accountant
Test Duration 30 mins 2 hrs 33 mins 29 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) Y Y N N
Remuneration (Quantitative Data Analysis)
Time taken to generate all payment slip 25 seconds 50 seconds 15 seconds 25 seconds
Time taken to update payment status 10 seconds 1 minute 15 seconds 10 seconds
Total number of clicks to finish this module 3 5 5 5
Total number of question asked 1 3 2 2
Total number of bug found 0 0 0 0
Claims (Quantitative Data Analysis)
Time taken to submit new claims 2 minutes 1 minute 1 minute 30 seconds
Time taken to validate submitted claims 1 minute 20 seconds 30 seconds 20 seconds
Time taken to approve/ reject submitted claims 10 seconds 10 seconds 5 seconds 10 seconds
Time taken to view claim status 5 seconds 10 seconds 5 seconds 6 seconds
Time taken to generate claim slip 5 seconds 10 seconds 10 seconds 25 seconds
Total number of clicks to finish this module 17 16 18 17
Total number of question asked 5 3 1 7
Total number of bug found 2 0 1 1
Leave(Quantitative Data Analysis)
Time taken to apply leave 2 minutes 1 minute 1 minute 25 seconds
Time taken to edit leave 30 seconds 40 seconds 1 minute 35 seconds
Time taken to approve/reject leave 20 seconds 30 seconds 30 seconds 10 seconds
Time taken to view leave that are being approved 10 seconds 30 seconds 10 seconds 5 seconds
Total number of clicks to finish this module 10 11 12 11
Total number of question asked 4 1 2 3
Total number of bug found 0 0 0 0


Category Pilot Participant Participant 1 Participant 2 Participant 3
Navigation/Login/Overall look
  • Ugly layout for navigation bar
  • Poor navigation bar where links are hard to find
  • Suggest to have vertical links instead
  • Suggest for navigation bar links to be clicked instead of hover
  • Mouse-over very sensitive for the navigation bar
  • Navigation bar too long
  • Layout is not very nice
Remuneration
  • Not a need to have prompt message for update payment status
  • Suggest to have the Month the payslip is on to be at the top of the payslip
  • Suggest to include basic pay in the pay slip
  • Payslip description is a bit vague
  • Include basic pay in pay slip
  • Basic salary not shown in payslip
Claims
  • Upload button is redundant
  • Unclear table headings
  • Unable to view claim submitted
  • Incorrect naming of button of "Generate Pay Slip", it should be "Generate Claim Slip"
  • Add or remove row button inserted at the bottom of table
  • Button is wrongly named. Should be “Generate claim slip” instead
  • Adding new row to submit another claim is not clear
  • The value for the button is wrong. It should be Generate Claim slip
Leave
  • Cannot find arrow on selection on calendar
  • Inconsistent spelling
  • System should calculate the number of days taken and make it more obvious
  • Suggest "Confirm" button to calculate balance if leave is approved
  • Radio button look like bullet point
  • Radio button should be white when unclicked
  • Page is very clustered with a lot of information
  • Button size not standardize
  • Calendar is not working properly
  • Too many things cramped into one page
  • Naming not good
  • Button size not standardize

User Test 2

Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Pilot Participant Participant 1 Participant 2 Participant 3 Participant 4
ZORA UT2 PP.JPG
ZORA UT2 Ang.JPG
ZORA UT2 LC.JPG
ZORA UT2 Ro.JPG
ZORA UT2 FR.JPG
General Data
Age 35 45 36 31 35
Occupation Business Development Manager HR Personnel Contract Manager (HM) Structural Draftsman Quantity Surveyor
Test Duration 1 hour 30 mins 1 hour 40 mins 1 hour 10 mins 1 hour 20 mins 1 hour 25 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) Y N N N N
Claims (Quantitative Data Analysis)
Time taken to submit new claims 59 seconds 2 minutes 36 seconds 2 minutes 30 seconds 1 minute 45 seconds 1 minute 35 seconds
Time taken to validate submitted claims 1 minute 27 seconds 2 minutes 9 seconds 1 minute 40 seconds 2 minutes 20 seconds 2 minutes 10 seconds
Time taken to edit payment method 41 seconds 1 minutes 18 seconds 24 seconds 45 seconds 36 seconds
Time taken to approve/ reject submitted claims 56 seconds 3 minutes 29 seconds 2 minutes 6 seconds 1 minute 30 seconds 2 minutes 20 second
Time taken to generate claim slip 26 seconds N/A N/A N/A N/A
Time taken to view claim history 33 seconds N/A N/A N/A N/A
Time taken to view claim status 12 seconds 21 seconds 22 seconds 25 seconds 28 seconds
Total number of clicks to finish this module 22 25 28 30 34
Total number of question asked 5 9 3 6 10
Total number of bug found 4 0 0 1 0
Leave (Quantitative Data Analysis)
Time taken to apply annual leave 33 seconds 2 minutes 16 seconds 1 minute 1 second 1 minute 15 seconds 1 minute 40 seconds
Time taken to edit leave 33 seconds 1 minute 32 seconds 22 seconds 35 seconds 40 seconds
Time taken to apply other leave N/A 5 minutes 8 seconds 1 minute 45 seconds 1 minute 35 seconds 2 minutes
Time taken to approve/reject leave 1 minute 27 seconds 3 minutes 18 seconds 1 minute 23 seconds 1 minute 1 minute 50 seconds
Time taken to view status of leave & cancel leave 20 seconds 59 seconds 51 seconds 1 minute 15 seconds 3 minutes
Time taken to view leave history 20 seconds 33 seconds 11 seconds 20 seconds 15 seconds
Time taken to record medical leave 1 minute 30 seconds 3 minutes 30 seconds 2 minutes 6 seconds 2 minutes 20 seconds 2 minutes 10 seconds
Total number of clicks to finish this module 64 65 58 62 60
Total number of question asked 4 11 1 6 10
Total number of bug found 1 1 1 1 0
Remuneration (Quantitative Data Analysis)
Time taken to view past payment record for employees N/A 22 seconds 10 seconds 18 seconds N/A
Time taken to view past payment record for workers 4 seconds 14 seconds 5 seconds 10 seconds N/A
Time taken to view past payment record for drivers 1 seconds 7 seconds 24 seconds 9 seconds N/A
Time taken to view past payment record for workers N/A 22 seconds 51 seconds 35 seconds N/A
Time taken to generate all payment record for employees N/A 22 seconds 51 seconds 35 seconds N/A
Time taken to generate all payment record for workers N/A 26 seconds 7 seconds 15 seconds N/A
Time taken to generate all payment record for drivers N/A 15 seconds 12 seconds 10 seconds N/A
Time taken to view personal payment record N/A 36 seconds 27 seconds 24 seconds N/A
Time taken to input timesheet for drivers 2 minutes 14 seconds 1 minute 31 seconds 1 minute 9 seconds 2 minute N/A
Time taken to input timesheet for workers N/A 3 minute 5 seconds 2 minute 19 seconds 2 minute 30 seconds N/A
Total number of clicks to finish this module 60 65 60 66 N/A
Total number of question asked 1 1 0 1 N/A
Total number of bug found 1 5 5 1 N/A
Performance Management & Appraisal (Quantitative Data Analysis)
Time taken to create employees appraisal 45 seconds 2 minutes 36 seconds 1 minute 47 seconds 2 minutes 1 minute 30 seconds
Time taken to approve/revise salary increment requests 23 seconds 26 seconds 29 seconds 1 minute 45 seconds 1 minute 15 seconds
Time taken to distribute bonus 50 seconds 2 minutes 38 seconds 2 minutes 24 seconds 50 seconds 45 seconds
Time taken to view scheduled increment/bonus N/A 19 seconds 8 seconds 10 seconds 12 seconds
Time taken to view increment history N/A 24 seconds 12 seconds 10 seconds 12 seconds
Total number of clicks to finish this module 43 40 45 41 39
Total number of question asked 2 7 3 3 3
Total number of bug found 0 0 0 0 0
System Administration (Quantitative Data Analysis)
Time taken to create new profile 4 minutes 3 seconds 7 minutes 37 seconds 3 minutes 53 seconds 6 minutes 5 minutes 20 seconds
Time taken to update profile 2 minutes 12 seconds 2 minutes 26 seconds 1 minute 30 seconds 1 minute 34 seconds
Time taken to manage default number of leave 35 seconds 1 minute 40 seconds 1 minute 20 seconds 45 seconds 35 seconds
Total number of clicks to finish this module 37 40 38 36 35
Total number of question asked 1 0 0 0 0
Total number of bug found 1 1 0 0 0
Category Pilot Participant Participant 1 Participant 2 Participant 3 Participant 4
Navigation/Login/Overall look
  • Fonts need to be bigger
  • User-friendly with nothing too overwhelming
  • Description and titles are simple and not too wordy
  • Find it difficult to access the pages
  • Generally fluid with navigation
  • Make the date format in a format everyone is used to (I.e. dd/mm/yyyy)
  • Application is helpful and easy to use
Claim
  • N/A
  • Had difficulty finding the receipt for viewing
  • Did not know the need to hit "Update" after editing the amount
  • Hard to locate the claim receipt
  • N/A
  • N/A
Leave
  • N/A
  • Did not notice the dropdown in Record Leave
  • Did not know how to upload more than 1 file
  • N/A
  • N/A
Remuneration
  • PDF can have more space
  • N/A
  • N/A
  • Minus sign in the pay slip for those value that is going to be deducted
  • Add date for every signature
  • N/A
Performance & Appraisal
  • N/A
  • N/A
  • N/A
  • Approve does not automatically save the new value, and also allow to revise the value just in case the value is wrong
  • The title for view scheduled increments/bonus seemed confusing
  • N/A
System Administration
  • N/A
  • Colour of username is available should not be red in color
  • Ajax is not working in Firefox
  • Have a dropdown to select department
  • Have validation for phone numbers and postal code
  • N/A
  • N/A

Response Plan (Implement)

S/N Functionality Feedback Proposed Solution Proposed Decision
Claim
1 Submit Claim User tend to click "Add Claim" instead of "Submit Claim" Will shift the "Add Claim" button to somewhere else" Implement
2 validate Claim & Approve/Reject Claim User cannot find where to view the claim receipt Make the "view" link bigger and more distinct Implement
Leave
1 Apply Leave User is unsure of how to upload more than 1 files while applying for leave that require supporting documents Have a short instruction before the upload file field (i.e. Hold down CTRL button to select multiple options) Implement
2 Apply Leave User is unsure of what to put in the "Date To" field to apply leave on the same day When user selects the date in "Date From" field, the same date should be repopulated in the "Date To" field Implement
3 Apply Leave/Record Leave User tends to type the date instead of choosing the date using the datepicker leading to the wrong format being input Have a short instruction before the "Date To" and "Date From" field to indicate the date format and give an example of it Implement
4 User should be able to view leave of their colleagues in the same department so that they can plan how to cover up leave Have a separate "View Leave" page for users to view leave of people from the same department Make the "view" link bigger and more distinct Implement
Remuneration
1 View Personal Payslip tab User feel that it is hard to navigate to view personal payment record link when it is at the bottom Change the arrangement for the links under remuneration tab Implement
2 View payslip User finds that the less part of the payslip can be more obvious with a "-" sign Include the "-" sign in the PDF payslip Implement
3 View payslip Instead of month only, should include year as well in the payslip Include the year Implement
Performance & Appraisal
1 Edit Salary Increment User does not click the "Save" button when revising the salary increment amont. As a result, the amount approved is not the revised amount. It is not obvious that the "Save" button should be clicked before submitting the form In the event the increment value is not saved, the user should not be allowed to approve the increment request Implement
2 View Scheduled Salary Increment/Scheduled Bonus Currently the "Scheduled salary increment" and "Scheduled Bonuses" tables reside on the same page ("View Scheduled Increments/Bonus"). This can be confusing to the user as he/she is unable to distinguish between scheduled salary increments and scheduled bonuses Change the colours of the tables to differentiate between scheduled salary increments and scheduled bonuses Implement
3 Submit Appraisal Placement of "Submit Appraisal" and "Clear" buttons. Button placements are different from other modules. Not intuitive and users may get confused Standardize position of buttons across modules Implement
4 Submit Appraisal When an appaisal has been submitted successfully, it is not obvious to the user Display a successful appraisal submission message Implement
System Administration
1 Manage Leave Change default no. of leave --> User unsure of change is successful cos there is no popup or anything Display a successful update message Implement
2 Create/Edit Profile Remind the user the DOB format to type. User unsure of what to type. The date thing also must select super long ago. super troublesome for them Display a date format beside the field Implement
Navigation
1 Navigation In the navigation bar, users tend to click on menu items which display sub-menus, this results in the menu options being minimize. Selection of sub-menu items require users to hover over the main menu items and then clicking the sub-menus items Enable the navigation bar to support both hover and click functionality. In both instances, the sub-menu will be displayed and the menu will not be minimized Implement
Login
1 Error message Error message for login is black in colour Error message for login should be red in colour Implement
2 Active status Account with inactive status can login Account with inactive status should not be able login Implement
Changes made from User Test 2 Feedbacks
Claims
550
Leave
550
Remuneration
550
Performance Management & Appraisal
550
Quantitative Feedback Analysis
Ease of Use
350
Overall Layout
350
User Test 2 Analysis

View Full Analysis.

Response Plan (Not Implementing)

S/N Functionality Feedback Reason for Not Implementing Proposed Decision
Leave
1 Approve/Reject Leave People in the department should have the authority of approving/rejecting the leave the user sends The sponsor of the project mentioned that the line manager should be the only one approving. He should know if anyone in the department rejects to that particular leave on a particular day in case of any project that arises. Hence, according to the business logic, only the line manager should have the authority to approve/reject leave Not Implementing
2 Apply Leave User takes quite a while to find the leave balance on the "Apply Leave" page As the task instruction explicitly mention to the tester to check on the leave balance, tester might not know that it can be checked in the "Apply Leave" page. Moreover, it is only 1 out of 5 of the user that mentioned that. However, everyone eventually managed to find the leave balance when they are applying leave Not Implementing
Remuneration
1 Payslip Suggestion: For payslip, put signature and date The payslip is followed closely to the current payslip that the company is using Not Implementing
Performance & Appraisal
1 Edit Appraisal User should have the option to edit Appraisal once submitted Client explicitly mentioned that Appraisals should not be edited once submitted Not Implementing
System Administration
1 Create/Edit Profile Creating user profile --> User tend to click Tab. Take note to rearrange input data values. E.g. Put "Given Name" beside "Family Name" cos that is where the cursor go when user click Tab. Do take note of this Not really important Not Implementing
2 Create/Edit Profile To input allowance --> amount for which allowance type? If user select more than 1 allowance type, the $500 is for which one? It suppose to be like this, is a combine amount Not Implementing
Login/Logout
1 Logout User takes a few secs to find logout button Attributed to lack of familiarity with the system user interface. Not Implementing

User Test 3

In this user test, we did the testing on 9 participants, of which 5 participants are the same person from user test 2. This is so that we are able to do comparison between the qualitative and quantitative results gathered from user test 2 with the results that we gathered in this user test 3. Due to the space constraints, we will be showing the results gathered from the 4 new participants. If you want to view more results from user test 3, please click on this ->

Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Participant 1 Participant 2 Participant 3 Participant 4
Zora UT3 Participant 6.png
ZORA UT3 CHING.JPG
ZORA UT3 RAS.jpg
ZORA UT3 STU.JPG
General Data
Age 28 30 30 20
Occupation Design Engineer Quantity Surveyor Work & Safety Health Officer Student Intern
Test Duration 1 hour 15 mins 1 hour 1 hour 4 mins 1 hour 30 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) N N N N
Claims (Quantitative Data Analysis)
Time taken to submit new claims 1 minute 27 seconds 37 seconds 1 minute 1 minute 54 seconds
Time taken to validate submitted claims 1 minute 21 seconds 1 minutes 14 seconds 2 minute 26 seconds 1 minutes 8 seconds
Time taken to edit payment method 22 seconds 15 seconds 37 seconds 43 seconds
Time taken to approve/ reject submitted claims 25 seconds 29 seconds 58 seconds 42 seconds
Time taken to generate claim slip 2 minutes 32 seconds 59 seconds 2 minutes 56 seconds 40 seconds
Time taken to view claim history 1 minute 14 seconds 30 seconds 41 seconds 20 seconds
Time taken to view claim status 27 seconds 20 seconds 51 seconds 9 seconds
Time taken to generate claim report 30 seconds 25 seconds 40 seconds 27 seconds
Total number of question asked 0 4 7 0
Total number of bug found 0 0 0 0
Leave (Quantitative Data Analysis)
Time taken to apply & edit annual leave 2 minutes 37 seconds 1 minutes 19 seconds 2 minute 36 second 1 minute 50 seconds
Time taken to apply other leave 2 minutes 55 seconds 2 minutes 1 minute 44 seconds 1 minute
Time taken to approve leave 2 minutes 36 seconds 37 seconds 3 minutes 15 seconds 2 minutes 10 seconds
Time taken to view status of leave & cancel leave 1 minute 24 seconds 55 seconds 1 minute 20 seconds 2 minute 4 seconds
Time taken to view leave history 10 seconds 10 seconds 1 minute 25 seconds 46 seconds
Time taken to record medical leave 2 minutes 47 seconds 1 minute 44 seconds 1 minute 45 seconds 2 minutes 25 seconds
Time taken to record unpaid leave & approve recorded leave 2 minutes 51 seconds 1 minutes 50 seconds 3 minutes 5 seconds 2 minutes 10 seconds
Time taken to generate leave report 15 seconds 56 seconds 2 minutes 35 seconds 15 seconds
Total number of question asked 1 2 6 3
Total number of bug found 0 0 0 0
Remuneration (Quantitative Data Analysis)
Time taken to generate all payment record for employees 1 minute 12 seconds 18 seconds 45 seconds 52 seconds
Time taken to view past payment record for employees 7 seconds 20 seconds 49 seconds 13 seconds
Time taken to view personal payment record 33 seconds 16 seconds 55 seconds 20 seconds
Time taken to input timesheet for workers 3 minutes 19 seconds 1 minutes 18 seconds 2 minutes 45 seconds 3 minutes 23 seconds
Time taken to input OT timesheet for drivers 1 minute 18 seconds 23 seconds 1 minute 47 seconds 3 minute 3 seconds
Time taken to view worker costing 13 seconds 5 seconds 1 minute 19 seconds 15 seconds
Total number of question asked 2 0 4 3
Total number of bug found 0 0 0 0
Performance Management & Appraisal (Quantitative Data Analysis)
Time taken to appraise employees 3 minutes 21 seconds 58 seconds 2 minute 21 seconds 3 minutes 19 seconds
Time taken to distribute bonus 1 minute 11 seconds 30 seconds 1 minute 55 seconds 52 seconds
Total number of question asked 0 0 4 1
Total number of bug found 0 0 0 0
Training & Development (Quantitative Data Analysis)
Time taken to propose training 1 minutes 25 seconds 35 seconds 3 minutes 1 minutes 57 second
Time taken to approve/reject training by HR 3 minutes 43 seconds 1 minute 55 seconds 6 minutes 3 seconds 2 minutes 55 seconds
Time taken to approve/reject training by MD 49 seconds 30 seconds 1 minute 44 seconds 55 seconds
Time taken to view proposed training status 26 seconds 14 seconds 47 seconds 1 minute 7 seconds
Time taken to view nominated training 1 minute 3 seconds 38 seconds 1 minute 5 seconds 21 seconds
Time taken to manage external training 1 minute 2 seconds 32 seconds 1 minute 35 seconds 1 minute 4 seconds
Total number of question asked 0 0 5 0
Total number of bug found 0 0 0 0
System Administration (Quantitative Data Analysis)
Time taken to manage default number of leave 1 minutes 41 seconds 24 seconds 1 minutes 22 seconds 31 seconds
Time taken to create new profile 5 minutes 58 seconds 5 minutes 14 seconds 3 minute 55 seconds 4 minute 59 seconds
Time taken to update profile 46 seconds 51 seconds 1 minute 7 seconds 1 minute 12 seconds
Total number of question asked 0 0 5 1
Total number of bug found 0 0 0 0