HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2013T2 Zora Testing"

From IS480
Jump to navigation Jump to search
Line 1,202: Line 1,202:
 
!style = "width: 22%; background:#66CCCC; color:black;"|'''Feedback'''
 
!style = "width: 22%; background:#66CCCC; color:black;"|'''Feedback'''
 
!style = "width: 23%; background:#66CCCC; color:black;"|'''Reason for Not Implementing'''
 
!style = "width: 23%; background:#66CCCC; color:black;"|'''Reason for Not Implementing'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Decision'''
  
 
|-
 
|-

Revision as of 21:56, 11 April 2014

Zora page header.png
Home Project Overview Project Management Documentation

Test Methodology

Zora test methodology.png

Zora Testing6.JPG

Test Plan

Zora test plan.png

Test Cases

Zora Testing14.JPG
Click here to view Login/Logout Test Case file.
Click here to view Remuneration Test Case file.
Click here to view Claim Test Case file.
Click here to view Leave Test Case file.
Click here to view Training & Development Test Case file.
Click here to view Performance Management & Appraisal Test Case file.
Click here to view Administration Test Case file.

Test Outcome

Heuristic Evaluation 1

Testers are the expert evaluators. They used the following heuristic and severity ratings, as shown below, to evaluate our application

Heuristics Severity Ratings

1. Visibility of system status

2. Match between system and the real world

3. User control and freedom

4. Consistency and standards

5. Error prevention

6. Recognition rather than recall

7. Flexibility and efficiency of use

8. Aesthetic and minimalist design

9. Help users recognize, diagnose, and recover from errors

10. Help and documentation

Not related to any heuristic

0 = I don't agree that this is a usability problem

1 = Cosmetic problem only

2 = Minor usability problem

3 = Major usability problem

4 = Usability catastrophe

Images taken during the evaluation

Yun Xi bringing Hui Shia through the remuneration part of the project
Mei Zhen explaining our prototype to Jing Yi
Yun Xi going through the prototype with Eng Sen
Yi Xian going through our prototype to Ken Khoon

Results obtained from the evaluation conducted

Zora Testing20.JPG






















User Test 1

Steps taken to conduct user testing 1
Post-Test
Identify:

  1. Goals of the test
  2. Scope of the test
  3. Participant Profile

Decides on:

  1. Testing set up (venue, time etc.)

Prepare:

  1. Testing items and documents

Recruit Testers

  1. Testers to read the study script


During-Test

  1. Give user informed form to testers
  2. Give task instructions to testers
  3. Give interview form to testers
  4. Observe and fill up Observation Form
  5. Give Questionnaire to testers to fill up while testing


Post-Test

  1. Consolidate Feedbacks
  2. Analyze Results
  3. Maintain a Response Plan
  4. Implement feedback in UT1 and test again in upcoming UT2


Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Pilot Participant Participant 1 Participant 2 Participant 3
IMG 4600.JPG
Zora Testing22.JPG
Zora Testing23.JPG
Zora Testing24.JPG
General Data
Age 35 50 23 26
Occupation Business Development Manager Administrator Student Accountant
Test Duration 30 mins 2 hrs 33 mins 29 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) Y Y N N
Remuneration (Quantitative Data Analysis)
Time taken to generate all payment slip 25 seconds 50 seconds 15 seconds 25 seconds
Time taken to update payment status 10 seconds 1 minute 15 seconds 10 seconds
Total number of clicks to finish this module 3 5 5 5
Total number of question asked 1 3 2 2
Total number of bug found 0 0 0 0
Claims (Quantitative Data Analysis)
Time taken to submit new claims 2 minutes 1 minute 1 minute 30 seconds
Time taken to validate submitted claims 1 minute 20 seconds 30 seconds 20 seconds
Time taken to approve/ reject submitted claims 10 seconds 10 seconds 5 seconds 10 seconds
Time taken to view claim status 5 seconds 10 seconds 5 seconds 6 seconds
Time taken to generate claim slip 5 seconds 10 seconds 10 seconds 25 seconds
Total number of clicks to finish this module 17 16 18 17
Total number of question asked 5 3 1 7
Total number of bug found 2 0 1 1
Leave(Quantitative Data Analysis)
Time taken to apply leave 2 minutes 1 minute 1 minute 25 seconds
Time taken to edit leave 30 seconds 40 seconds 1 minute 35 seconds
Time taken to approve/reject leave 20 seconds 30 seconds 30 seconds 10 seconds
Time taken view leave that are being approved 10 seconds 30 seconds 10 seconds 5 seconds
Total number of clicks to finish this module 10 11 12 11
Total number of question asked 4 1 2 3
Total number of bug found 0 0 0 0


Category Pilot Participant Participant 1 Participant 2 Participant 3
Navigation/Login/Overall look
  • Ugly layout for navigation bar
  • Poor navigation bar where links are hard to find
  • Suggest to have vertical links instead
  • Suggest for navigation bar links to be clicked instead of hover
  • Mouse-over very sensitive for the navigation bar
  • Navigation bar too long
  • Layout is not very nice
Remuneration
  • Not a need to have prompt message for update payment status
  • Suggest to have the Month the payslip is on to be at the top of the payslip
  • Suggest to include basic pay in the pay slip
  • Payslip description is a bit vague
  • Include basic pay in pay slip
  • Basic salary not shown in payslip
Claims
  • Upload button is redundant
  • Unclear table headings
  • Unable to view claim submitted
  • Incorrect naming of button of "Generate Pay Slip", it should be "Generate Claim Slip"
  • Add or remove row button inserted at the bottom of table
  • Button is wrongly named. Should be “Generate claim slip” instead
  • Adding new row to submit another claim is not clear
  • The value for the button is wrong. It should be Generate Claim slip
Leave
  • Cannot find arrow on selection on calendar
  • Inconsistent spelling
  • System should calculate the number of days taken and make it more obvious
  • Suggest "Confirm" button to calculate balance if leave is approved
  • Radio button look like bullet point
  • Radio button should be white when unclicked
  • Page is very clustered with a lot of information
  • Button size not standardize
  • Calendar is not working properly
  • Too many things cramped into one page
  • Naming not good
  • Button size not standardize

User Test 2

Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Pilot Participant Participant 1 Participant 2 Participant 3 Participant 4
ZORA UT2 PP.JPG
ZORA UT2 Ang.JPG
ZORA UT2 LC.JPG
ZORA UT2 Ro.JPG
ZORA UT2 FR.JPG
General Data
Age 35 45 36 31 35
Occupation Business Development Manager HR Personnel Contract Manager (HM) Structural Draftsman Quantity Surveyor
Test Duration 1 hour 30 mins 1 hour 40 mins 1 hour 10 mins 1 hour 20 mins 1 hour 25 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) Y N N N N
Claims (Quantitative Data Analysis)
Time taken to submit new claims 59 minutes 2 minutes 36 seconds 2 minutes 30 seconds 1 minute 45 seconds 1 minute 35 seconds
Time taken to validate submitted claims 1 minute 27 seconds 2 minutes 9 seconds 1 minute 40 seconds 2 minutes 20 seconds 2 minutes 10 seconds
Time taken to edit payment method 41 seconds 1 minutes 18 seconds 24 seconds 45 seconds 36 seconds
Time taken to approve/ reject submitted claims 56 seconds 3 minutes 29 seconds 2 minutes 6 seconds 1 minute 30 seconds 2 minutes 20 second
Time taken to generate claim slip 26 seconds N/A N/A N/A N/A
Time taken to view claim history 33 seconds N/A N/A N/A N/A
Time taken to view claim status 12 seconds 21 seconds 22 seconds 25 seconds 28 seconds
Total number of clicks to finish this module 22 25 28 30 34
Total number of question asked 5 9 3 6 10
Total number of bug found 4 0 0 1 0
Leave (Quantitative Data Analysis)
Time taken to apply annual leave 33 seconds 2 minutes 16 seconds 1 minute 1 second 1 minute 15 seconds 1 minute 40 seconds
Time taken to edit leave 33 seconds 1 minute 32 seconds 22 seconds 35 seconds 40 seconds
Time taken to apply other leave N/A 5 minutes 8 seconds 1 minute 45 seconds 1 minute 35 seconds 2 minutes
Time taken to approve/reject leave 1 minute 27 seconds 3 minutes 18 seconds 1 minute 23 seconds 1 minute 1 minute 50 seconds
Time taken view status of leave & cancel leave 20 seconds 59 seconds 51 seconds 1 minute 15 seconds 3 minutes
Time taken view leave history 20 seconds 33 seconds 11 seconds 20 seconds 15 seconds
Time taken record medical leave 1 minute 30 seconds 3 minutes 30 seconds 2 minutes 6 seconds 2 minutes 20 seconds 2 minutes 10 seconds
Total number of clicks to finish this module 64 65 58 62 60
Total number of question asked 4 11 1 6 10
Total number of bug found 1 1 1 1 0
Remuneration (Quantitative Data Analysis)
Time taken to view past payment record for employees N/A 22 seconds 10 seconds 18 seconds N/A
Time taken to view past payment record for workers 4 seconds 14 seconds 5 seconds 10 seconds N/A
Time taken to view past payment record for drivers 1 seconds 7 seconds 24 seconds 9 seconds N/A
Time taken to view past payment record for workers N/A 22 seconds 51 seconds 35 seconds N/A
Time taken to generate all payment record for employees N/A 22 seconds 51 seconds 35 seconds N/A
Time taken to generate all payment record for workers N/A 26 seconds 7 seconds 15 seconds N/A
Time taken to generate all payment record for drivers N/A 15 seconds 12 seconds 10 seconds N/A
Time taken to view personal payment record N/A 36 seconds 27 seconds 24 seconds N/A
Time taken to input timesheet for drivers 2 minutes 14 seconds 1 minute 31 seconds 1 minute 9 seconds 2 minute N/A
Time taken to input timesheet for workers N/A 3 minute 5 seconds 2 minute 19 seconds 2 minute 30 seconds N/A
Total number of clicks to finish this module 60 65 60 66 N/A
Total number of question asked 1 1 0 1 N/A
Total number of bug found 1 5 5 1 N/A
Performance Management & Appraisal (Quantitative Data Analysis)
Time taken to create employees appraisal 45 seconds 2 minutes 36 seconds 1 minute 47 seconds 2 minutes 1 minute 30 seconds
Time taken to approve/revise salary increment requests 23 seconds 26 seconds 29 seconds 1 minute 45 seconds 1 minute 15 seconds
Time taken to distribute bonus 50 seconds 2 minutes 38 seconds 2 minutes 24 seconds 50 seconds 45 seconds
Time taken to view scheduled increment/bonus N/A 19 seconds 8 seconds 10 seconds 12 seconds
Time taken to view increment history N/A 24 seconds 12 seconds 10 seconds 12 seconds
Total number of clicks to finish this module 43 40 45 41 39
Total number of question asked 2 7 3 3 3
Total number of bug found 0 0 0 0 0
System Administration (Quantitative Data Analysis)
Time taken to create new profile 4 minutes 3 seconds 7 minutes 37 seconds 3 minutes 53 seconds 6 minutes 5 minutes 20 seconds
Time taken to update profile 2 minutes 12 seconds 2 minutes 26 seconds 1 minute 30 seconds 1 minute 34 seconds
Time taken to manage default number of leave 35 seconds 1 minute 40 seconds 1 minute 20 seconds 45 seconds 35 seconds
Total number of clicks to finish this module 37 40 38 36 35
Total number of question asked 1 0 0 0 0
Total number of bug found 1 1 0 0 0
Category Pilot Participant Participant 1 Participant 2 Participant 3 Participant 4
Navigation/Login/Overall look
  • Fonts need to be bigger
  • User-friendly with nothing too overwhelming
  • Description and titles are simple and not too wordy
  • Find it difficult to access the pages
  • Generally fluid with navigation
  • Make the date format in a format everyone is used to (I.e. dd/mm/yyyy)
  • Application is helpful and easy to use
Claim
  • N/A
  • Had difficulty finding the receipt for viewing
  • Did not know the need to hit "Update" after editing the amount
  • Hard to locate the claim receipt
  • N/A
  • N/A
Leave
  • N/A
  • Did not notice the dropdown in Record Leave
  • Did not know how to upload more than 1 file
  • N/A
  • N/A
Remuneration
  • PDF can have more space
  • N/A
  • N/A
  • Minus sign in the pay slip for those value that is going to be deducted
  • Add date for every signature
  • N/A
Performance & Appraisal
  • N/A
  • N/A
  • N/A
  • Approve does not automatically save the new value, and also allow to revise the value just in case the value is wrong
  • The title for view scheduled increments/bonus seemed confusing
  • N/A
System Administration
  • N/A
  • Colour of username is available should not be red in color
  • Ajax is not working in Firefox
  • Have a dropdown to select department
  • Have validation for phone numbers and postal code
  • N/A
  • N/A

Response Plan (Implement)

S/N Functionality Feedback Proposed Solution Proposed Decision
Claim
1 Submit Claim User tend to click "Add Claim" instead of "Submit Claim" Will shift the "Add Claim" button to somewhere else" Implement
2 validate Claim & Approve/Reject Claim User cannot find where to view the claim receipt Make the "view" link bigger and more distinct Implement
Leave
1 Apply Leave User is unsure of how to upload more than 1 files while applying for leave that require supporting documents Have a short instruction before the upload file field (i.e. Hold down CTRL button to select multiple options) Implement
2 Apply Leave User is unsure of what to put in the "Date To" field to apply leave on the same day When user selects the date in "Date From" field, the same date should be repopulated in the "Date To" field Implement
3 Apply Leave/Record Leave User tends to type the date instead of choosing the date using the datepicker leading to the wrong format being input Have a short instruction before the "Date To" and "Date From" field to indicate the date format and give an example of it Implement
4 User should be able to view leave of their colleagues in the same department so that they can plan how to cover up leave Have a separate "View Leave" page for users to view leave of people from the same department Make the "view" link bigger and more distinct Implement
Remuneration
1 View Personal Payslip tab User feel that it is hard to navigate to view personal payment record link when it is at the bottom Change the arrangement for the links under remuneration tab Implement
2 View payslip User finds that the less part of the payslip can be more obvious with a "-" sign Include the "-" sign in the PDF payslip Implement
3 View payslip Instead of month only, should include year as well in the payslip Include the year Implement
Performance & Appraisal
1 Edit Salary Increment User does not click the "Save" button when revising the salary increment amont. As a result, the amount approved is not the revised amount. It is not obvious that the "Save" button should be clicked before submitting the form In the event the increment value is not saved, the user should not be allowed to approve the increment request Implement
2 View Scheduled Salary Increment/Scheduled Bonus Currently the "Scheduled salary increment" and "Scheduled Bonuses" tables reside on the same page ("View Scheduled Increments/Bonus"). This can be confusing to the user as he/she is unable to distinguish between scheduled salary increments and scheduled bonuses Change the colours of the tables to differentiate between scheduled salary increments and scheduled bonuses Implement
3 Submit Appraisal Placement of "Submit Appraisal" and "Clear" buttons. Button placements are different from other modules. Not intuitive and users may get confused Standardize position of buttons across modules Implement
4 Submit Appraisal When an appaisal has been submitted successfully, it is not obvious to the user Display a successful appraisal submission message Implement
System Administration
1 Manage Leave Change default no. of leave --> User unsure of change is successful cos there is no popup or anything Display a successful update message Implement
2 Create/Edit Profile Remind the user the DOB format to type. User unsure of what to type. The date thing also must select super long ago. super troublesome for them Display a date format beside the field Implement
Navigation
1 Navigation In the navigation bar, users tend to click on menu items which display sub-menus, this results in the menu options being minimize. Selection of sub-menu items require users to hover over the main menu items and then clicking the sub-menus items Enable the navigation bar to support both hover and click functionality. In both instances, the sub-menu will be displayed and the menu will not be minimized Implement
Login
1 Error message Error message for login is black in colour Error message for login should be red in colour Implement
2 Active status Account with inactive status can login Account with inactive status should not be able login Implement
Changes made from User Test 2 Feedbacks
Claims
550
Leave
550
Remuneration
550
Performance Management & Appraisal
550
Quantitative Feedback Analysis
Ease of Use
350
Overall Layout
350
User Test 2 Analysis

View Full Analysis.

Response Plan (Not Implementing)

S/N Functionality Feedback Reason for Not Implementing Proposed Decision
Leave
1 Approve/Reject Leave People in the department should have the authority of approving/rejecting the leave the user sends The sponsor of the project mentioned that the line manager should be the only one approving. He should know if anyone in the department rejects to that particular leave on a particular day in case of any project that arises. Hence, according to the business logic, only the line manager should have the authority to approve/reject leave Not Implementing
2 Apply Leave User takes quite a while to find the leave balance on the "Apply Leave" page As the task instruction explicitly mention to the tester to check on the leave balance, tester might not know that it can be checked in the "Apply Leave" page. Moreover, it is only 1 out of 5 of the user that mentioned that. However, everyone eventually managed to find the leave balance when they are applying leave Not Implementing
Remuneration
1 Payslip Suggestion: For payslip, put signature and date The payslip is followed closely to the current payslip that the company is using Not Implementing
Performance & Appraisal
1 Create/Edit Profile User should have the option to edit Appraisal once submitted Client explicitly mentioned that Appraisals should not be edited once submitted Not Implementing