|
Home
|
Project Overview
|
Project Management
|
Documentation
|
|
|
|
|
Test Methodology
Test Plan
Test Cases
Click here to view Login/Logout Test Case file.
Click here to view Remuneration Test Case file.
Click here to view Claim Test Case file.
Click here to view Leave Test Case file.
Click here to view Training & Development Test Case file.
Click here to view Performance Management & Appraisal Test Case file.
Click here to view Administration Test Case file.
Test Outcome
Heuristic Evaluation 1
Testers are the expert evaluators. They used the following heuristic and severity ratings, as shown below, to evaluate our application
Heuristics
|
Severity Ratings
|
1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation
Not related to any heuristic
|
0 = I don't agree that this is a usability problem
1 = Cosmetic problem only
2 = Minor usability problem
3 = Major usability problem
4 = Usability catastrophe
|
Images taken during the evaluation
Yun Xi bringing Hui Shia through the remuneration part of the project |
Mei Zhen explaining our prototype to Jing Yi |
Yun Xi going through the prototype with Eng Sen
|
Yi Xian going through our prototype to Ken Khoon
|
Results obtained from the evaluation conducted
User Test 1
Steps taken to conduct user testing 1
Post-Test
Identify:
- Goals of the test
- Scope of the test
- Participant Profile
Decides on:
- Testing set up (venue, time etc.)
Prepare:
- Testing items and documents
Recruit Testers
- Testers to read the study script
During-Test
- Give user informed form to testers
- Give task instructions to testers
- Give interview form to testers
- Observe and fill up Observation Form
- Give Questionnaire to testers to fill up while testing
Post-Test
- Consolidate Feedbacks
- Analyze Results
- Maintain a Response Plan
- Implement feedback in UT1 and test again in upcoming UT2
Results obtained from the testing conducted
The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:
Data
|
Pilot Participant
|
Participant 1
|
Participant 2
|
Participant 3
|
|
|
|
|
|
General Data
|
Age
|
35
|
50
|
23
|
26
|
Occupation
|
Business Development Manager
|
Administrator
|
Student
|
Accountant
|
Test Duration
|
30 mins
|
2 hrs
|
33 mins
|
29 mins
|
Browser
|
Google Chrome
|
Google Chrome
|
Google Chrome
|
Google Chrome
|
Experienced in HRMS (Y/N)
|
Y
|
Y
|
N
|
N
|
Remuneration (Quantitative Data Analysis)
|
Time taken to generate all payment slip
|
25 seconds
|
50 seconds
|
15 seconds
|
25 seconds
|
Time taken to update payment status
|
10 seconds
|
1 minute
|
15 seconds
|
10 seconds
|
Total number of clicks to finish this module
|
3
|
5
|
5
|
5
|
Total number of question asked
|
1
|
3
|
2
|
2
|
Total number of bug found
|
0
|
0
|
0
|
0
|
Claims (Quantitative Data Analysis)
|
Time taken to submit new claims
|
2 minutes
|
1 minute
|
1 minute
|
30 seconds
|
Time taken to validate submitted claims
|
1 minute
|
20 seconds
|
30 seconds
|
20 seconds
|
Time taken to approve/ reject submitted claims
|
10 seconds
|
10 seconds
|
5 seconds
|
10 seconds
|
Time taken to view claim status
|
5 seconds
|
10 seconds
|
5 seconds
|
6 seconds
|
Time taken to generate claim slip
|
5 seconds
|
10 seconds
|
10 seconds
|
25 seconds
|
Total number of clicks to finish this module
|
17
|
16
|
18
|
17
|
Total number of question asked
|
5
|
3
|
1
|
7
|
Total number of bug found
|
2
|
0
|
1
|
1
|
Leave(Quantitative Data Analysis)
|
Time taken to apply leave
|
2 minutes
|
1 minute
|
1 minute
|
25 seconds
|
Time taken to edit leave
|
30 seconds
|
40 seconds
|
1 minute
|
35 seconds
|
Time taken to approve/reject leave
|
20 seconds
|
30 seconds
|
30 seconds
|
10 seconds
|
Time taken view leave that are being approved
|
10 seconds
|
30 seconds
|
10 seconds
|
5 seconds
|
Total number of clicks to finish this module
|
10
|
11
|
12
|
11
|
Total number of question asked
|
4
|
1
|
2
|
3
|
Total number of bug found
|
0
|
0
|
0
|
0
|
Category
|
Pilot Participant
|
Participant 1
|
Participant 2
|
Participant 3
|
Navigation/Login/Overall look
|
- Ugly layout for navigation bar
|
- Poor navigation bar where links are hard to find
- Suggest to have vertical links instead
- Suggest for navigation bar links to be clicked instead of hover
|
- Mouse-over very sensitive for the navigation bar
- Navigation bar too long
|
|
Remuneration
|
- Not a need to have prompt message for update payment status
|
- Suggest to have the Month the payslip is on to be at the top of the payslip
- Suggest to include basic pay in the pay slip
|
- Payslip description is a bit vague
- Include basic pay in pay slip
|
- Basic salary not shown in payslip
|
Claims
|
- Upload button is redundant
|
- Unclear table headings
- Unable to view claim submitted
- Incorrect naming of button of "Generate Pay Slip", it should be "Generate Claim Slip"
|
- Add or remove row button inserted at the bottom of table
- Button is wrongly named. Should be “Generate claim slip” instead
|
- Adding new row to submit another claim is not clear
- The value for the button is wrong. It should be Generate Claim slip
|
Leave
|
- Cannot find arrow on selection on calendar
- Inconsistent spelling
|
- System should calculate the number of days taken and make it more obvious
- Suggest "Confirm" button to calculate balance if leave is approved
- Radio button look like bullet point
|
- Radio button should be white when unclicked
- Page is very clustered with a lot of information
- Button size not standardize
|
- Calendar is not working properly
- Too many things cramped into one page
- Naming not good
- Button size not standardize
|
User Test 2
Results obtained from the testing conducted
The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:
Data
|
Pilot Participant
|
Participant 1
|
Participant 2
|
Participant 3
|
Participant 4
|
|
|
|
|
|
|
General Data
|
Age
|
35
|
45
|
36
|
31
|
35
|
Occupation
|
Business Development Manager
|
HR Personnel
|
Contract Manager (HM)
|
Structural Draftsman
|
Quantity Surveyor
|
Test Duration
|
1 hour 30 mins
|
1 hour 40 mins
|
1 hour 10 mins
|
1 hour 20 mins
|
1 hour 25 mins
|
Browser
|
Google Chrome
|
Google Chrome
|
Google Chrome
|
Google Chrome
|
Google Chrome
|
Experienced in HRMS (Y/N)
|
Y
|
N
|
N
|
N
|
N
|
Claims (Quantitative Data Analysis)
|
Time taken to submit new claims
|
59 minutes
|
2 minutes 36 seconds
|
2 minutes 30 seconds
|
1 minute 45 seconds
|
1 minute 35 seconds
|
Time taken to validate submitted claims
|
1 minute 27 seconds
|
2 minutes 9 seconds
|
1 minute 40 seconds
|
2 minutes 20 seconds
|
2 minutes 10 seconds
|
Time taken to edit payment method
|
41 seconds
|
1 minutes 18 seconds
|
24 seconds
|
45 seconds
|
36 seconds
|
Time taken to approve/ reject submitted claims
|
56 seconds
|
3 minutes 29 seconds
|
2 minutes 6 seconds
|
1 minute 30 seconds
|
2 minutes 20 second
|
Time taken to generate claim slip
|
26 seconds
|
N/A
|
N/A
|
N/A
|
N/A
|
Time taken to view claim history
|
33 seconds
|
N/A
|
N/A
|
N/A
|
N/A
|
Time taken to view claim status
|
12 seconds
|
21 seconds
|
22 seconds
|
25 seconds
|
28 seconds
|
Total number of clicks to finish this module
|
22
|
25
|
28
|
30
|
34
|
Total number of question asked
|
5
|
9
|
3
|
6
|
10
|
Total number of bug found
|
4
|
0
|
0
|
1
|
0
|
Leave (Quantitative Data Analysis)
|
Time taken to apply annual leave
|
33 seconds
|
2 minutes 16 seconds
|
1 minute 1 second
|
1 minute 15 seconds
|
1 minute 40 seconds
|
Time taken to edit leave
|
33 seconds
|
1 minute 32 seconds
|
22 seconds
|
35 seconds
|
40 seconds
|
Time taken to apply other leave
|
N/A
|
5 minutes 8 seconds
|
1 minute 45 seconds
|
1 minute 35 seconds
|
2 minutes
|
Time taken to approve/reject leave
|
1 minute 27 seconds
|
3 minutes 18 seconds
|
1 minute 23 seconds
|
1 minute
|
1 minute 50 seconds
|
Time taken view status of leave & cancel leave
|
20 seconds
|
59 seconds
|
51 seconds
|
1 minute 15 seconds
|
3 minutes
|
Time taken view leave history
|
20 seconds
|
33 seconds
|
11 seconds
|
20 seconds
|
15 seconds
|
Time taken record medical leave
|
1 minute 30 seconds
|
3 minutes 30 seconds
|
2 minutes 6 seconds
|
2 minutes 20 seconds
|
2 minutes 10 seconds
|
Total number of clicks to finish this module
|
64
|
65
|
58
|
62
|
60
|
Total number of question asked
|
4
|
11
|
1
|
6
|
10
|
Total number of bug found
|
1
|
1
|
1
|
1
|
0
|
Remuneration (Quantitative Data Analysis)
|
Time taken to view past payment record for employees
|
N/A
|
22 seconds
|
10 seconds
|
18 seconds
|
N/A
|
Time taken to view past payment record for workers
|
4 seconds
|
14 seconds
|
5 seconds
|
10 seconds
|
N/A
|
Time taken to view past payment record for drivers
|
1 seconds
|
7 seconds
|
24 seconds
|
9 seconds
|
N/A
|
Time taken to view past payment record for workers
|
N/A
|
22 seconds
|
51 seconds
|
35 seconds
|
N/A
|
Time taken to generate all payment record for employees
|
N/A
|
22 seconds
|
51 seconds
|
35 seconds
|
N/A
|
Time taken to generate all payment record for workers
|
N/A
|
26 seconds
|
7 seconds
|
15 seconds
|
N/A
|
Time taken to generate all payment record for drivers
|
N/A
|
15 seconds
|
12 seconds
|
10 seconds
|
N/A
|
Time taken to view personal payment record
|
N/A
|
36 seconds
|
27 seconds
|
24 seconds
|
N/A
|
Time taken to input timesheet for drivers
|
2 minutes 14 seconds
|
1 minute 31 seconds
|
1 minute 9 seconds
|
2 minute
|
N/A
|
Time taken to input timesheet for workers
|
N/A
|
3 minute 5 seconds
|
2 minute 19 seconds
|
2 minute 30 seconds
|
N/A
|
Total number of clicks to finish this module
|
60
|
65
|
60
|
66
|
N/A
|
Total number of question asked
|
1
|
1
|
0
|
1
|
N/A
|
Total number of bug found
|
1
|
5
|
5
|
1
|
N/A
|
Performance Management & Appraisal (Quantitative Data Analysis)
|
Time taken to create employees appraisal
|
45 seconds
|
2 minutes 36 seconds
|
1 minute 47 seconds
|
2 minutes
|
1 minute 30 seconds
|
Time taken to approve/revise salary increment requests
|
23 seconds
|
26 seconds
|
29 seconds
|
1 minute 45 seconds
|
1 minute 15 seconds
|
Time taken to distribute bonus
|
50 seconds
|
2 minutes 38 seconds
|
2 minutes 24 seconds
|
50 seconds
|
45 seconds
|
Time taken to view scheduled increment/bonus
|
N/A
|
19 seconds
|
8 seconds
|
10 seconds
|
12 seconds
|
Time taken to view increment history
|
N/A
|
24 seconds
|
12 seconds
|
10 seconds
|
12 seconds
|
Total number of clicks to finish this module
|
43
|
40
|
45
|
41
|
39
|
Total number of question asked
|
2
|
7
|
3
|
3
|
3
|
Total number of bug found
|
0
|
0
|
0
|
0
|
0
|
System Administration (Quantitative Data Analysis)
|
Time taken to create new profile
|
4 minutes 3 seconds
|
7 minutes 37 seconds
|
3 minutes 53 seconds
|
6 minutes
|
5 minutes 20 seconds
|
Time taken to update profile
|
2 minutes 12 seconds
|
2 minutes 26 seconds
|
1 minute 30 seconds
|
1 minute
|
34 seconds
|
Time taken to manage default number of leave
|
35 seconds
|
1 minute 40 seconds
|
1 minute 20 seconds
|
45 seconds
|
35 seconds
|
Total number of clicks to finish this module
|
37
|
40
|
38
|
36
|
35
|
Total number of question asked
|
1
|
0
|
0
|
0
|
0
|
Total number of bug found
|
1
|
1
|
0
|
0
|
0
|
Category
|
Pilot Participant
|
Participant 1
|
Participant 2
|
Participant 3
|
Participant 4
|
Navigation/Login/Overall look
|
- Fonts need to be bigger
- User-friendly with nothing too overwhelming
- Description and titles are simple and not too wordy
|
- Find it difficult to access the pages
|
- Generally fluid with navigation
|
- Make the date format in a format everyone is used to (I.e. dd/mm/yyyy)
|
- Application is helpful and easy to use
|
Claim
|
|
- Had difficulty finding the receipt for viewing
- Did not know the need to hit "Update" after editing the amount
|
- Hard to locate the claim receipt
|
|
|
Leave
|
|
- Did not notice the dropdown in Record Leave
|
- Did not know how to upload more than 1 file
|
|
|
Remuneration
|
|
|
|
- Minus sign in the pay slip for those value that is going to be deducted
- Add date for every signature
|
|
Performance & Appraisal
|
|
|
|
- Approve does not automatically save the new value, and also allow to revise the value just in case the value is wrong
- The title for view scheduled increments/bonus seemed confusing
|
|
System Administration
|
|
- Colour of username is available should not be red in color
|
- Ajax is not working in Firefox
- Have a dropdown to select department
- Have validation for phone numbers and postal code
|
|
|
Response Plan (Implement)
S/N
|
Functionality
|
Feedback
|
Proposed Solution
|
Proposed Decision
|
Claim
|
1
|
Submit Claim
|
User tend to click "Add Claim" instead of "Submit Claim"
|
Will shift the "Add Claim" button to somewhere else"
|
Implement
|
2
|
validate Claim & Approve/Reject Claim
|
User cannot find where to view the claim receipt
|
Make the "view" link bigger and more distinct
|
Implement
|
Leave
|
1
|
Apply Leave
|
User is unsure of how to upload more than 1 files while applying for leave that require supporting documents
|
Have a short instruction before the upload file field (i.e. Hold down CTRL button to select multiple options)
|
Implement
|
2
|
Apply Leave
|
User is unsure of what to put in the "Date To" field to apply leave on the same day
|
When user selects the date in "Date From" field, the same date should be repopulated in the "Date To" field
|
Implement
|
3
|
Apply Leave/Record Leave
|
User tends to type the date instead of choosing the date using the datepicker leading to the wrong format being input
|
Have a short instruction before the "Date To" and "Date From" field to indicate the date format and give an example of it
|
Implement
|
4
|
User should be able to view leave of their colleagues in the same department so that they can plan how to cover up leave
|
Have a separate "View Leave" page for users to view leave of people from the same department
|
Make the "view" link bigger and more distinct
|
Implement
|
Remuneration
|
1
|
View Personal Payslip tab
|
User feel that it is hard to navigate to view personal payment record link when it is at the bottom
|
Change the arrangement for the links under remuneration tab
|
Implement
|
2
|
View payslip
|
User finds that the less part of the payslip can be more obvious with a "-" sign
|
Include the "-" sign in the PDF payslip
|
Implement
|
3
|
View payslip
|
Instead of month only, should include year as well in the payslip
|
Include the year
|
Implement
|
Performance & Appraisal
|
1
|
Edit Salary Increment
|
User does not click the "Save" button when revising the salary increment amont. As a result, the amount approved is not the revised amount. It is not obvious that the "Save" button should be clicked before submitting the form
|
In the event the increment value is not saved, the user should not be allowed to approve the increment request
|
Implement
|
2
|
View Scheduled Salary Increment/Scheduled Bonus
|
Currently the "Scheduled salary increment" and "Scheduled Bonuses" tables reside on the same page ("View Scheduled Increments/Bonus"). This can be confusing to the user as he/she is unable to distinguish between scheduled salary increments and scheduled bonuses
|
Change the colours of the tables to differentiate between scheduled salary increments and scheduled bonuses
|
Implement
|
3
|
Submit Appraisal
|
Placement of "Submit Appraisal" and "Clear" buttons. Button placements are different from other modules. Not intuitive and users may get confused
|
Standardize position of buttons across modules
|
Implement
|
4
|
Submit Appraisal
|
When an appaisal has been submitted successfully, it is not obvious to the user
|
Display a successful appraisal submission message
|
Implement
|
System Administration
|
1
|
Manage Leave
|
Change default no. of leave --> User unsure of change is successful cos there is no popup or anything
|
Display a successful update message
|
Implement
|
2
|
Create/Edit Profile
|
Remind the user the DOB format to type. User unsure of what to type. The date thing also must select super long ago. super troublesome for them
|
Display a date format beside the field
|
Implement
|
Navigation
|
1
|
Navigation
|
In the navigation bar, users tend to click on menu items which display sub-menus, this results in the menu options being minimize. Selection of sub-menu items require users to hover over the main menu items and then clicking the sub-menus items
|
Enable the navigation bar to support both hover and click functionality. In both instances, the sub-menu will be displayed and the menu will not be minimized
|
Implement
|
Login
|
1
|
Error message
|
Error message for login is black in colour
|
Error message for login should be red in colour
|
Implement
|
2
|
Active status
|
Account with inactive status can login
|
Account with inactive status should not be able login
|
Implement
|