HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2013T2 Zora Testing"

From IS480
Jump to navigation Jump to search
(New page: {| class="wikitable" |- ! <font color="#069" face="">'''Home'''</font> !! Project Overview !! Project Management !!Documentation |- | [[IS480_Team_wiki:_20...)
 
 
(37 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{| class="wikitable"
+
{| style="background: #cedad1" cellpadding="0" height: 200px"  width="1050px"
 +
! scope="col" width="100" bgcolor="#cedad1" rowspan="2"| [[Image:zora_page_header.png|200px|left]]
 +
! scope="col" width="200" bgcolor="#a7b1aa"|[[IS480_Team_wiki:_2013T2_Zora|<font color="white" face="">'''Home'''</font>]]
 +
! scope="col" width="230" bgcolor="#a7b1aa"|<font color="white">Project Overview</font>
 +
! scope="col" width="230" bgcolor="#a7b1aa"|<font color="white">Project Management</font>
 +
! scope="col" width="250" bgcolor="#a7b1aa"|<font color="white">Documentation</font>
 +
|- align="left"
 +
|bgcolor="#cedad1" valign="top"|
 +
*[[IS480_Team_wiki:_2013T2_Zora#Notifications|<font color="#069" face="">'''Notifications'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora#Progress|<font color="#069" face="">'''Progress'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora#Milestones|<font color="#069" face="">'''Milestones'''</font>]]
 +
|bgcolor="#cedad1" valign="top"|
 +
*[[IS480_Team_wiki:_2013T2_Zora_Project_Description|<font color="#069" face="">'''Project Description'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Stakeholders|<font color="#069" face="">'''Stakeholders'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_User_Roles|<font color="#069" face="">'''User Roles'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Modules|<font color="#069" face="">'''Modules'''</font>]] 
 +
*[[IS480_Team_wiki:_2013T2_Zora_X_Factor|<font color="#069" face="">'''X Factor'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Team_Zora|<font color="#069" face="">'''Team Zora'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Learning_Outcomes|<font color="#069" face="">'''Learning Outcomes'''</font>]]
 +
|bgcolor="#cedad1" valign="top"|
 +
*[[IS480_Team_wiki:_2013T2_Zora_Project_Schedule|<font color="#069" face="">'''Project Schedule'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Framework|<font color="#069" face="">'''Framework'''</font>]] 
 +
*[[IS480_Team_wiki:_2013T2_Zora_Scope|<font color="#069" face="">'''Scope'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Metrics|<font color="#069" face="">'''Metrics'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Risks|<font color="#069" face="">'''Risks'''</font>]]
 +
|bgcolor="#cedad1" valign="top"|
 +
*[[IS480_Team_wiki:_2013T2_Zora_Meeting_Minutes|<font color="#069" face="">'''Meeting Minutes'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Business_Requirements|<font color="#069" face="">'''Business Requirements'''</font>]] 
 +
*[[IS480_Team_wiki:_2013T2_Zora_System_Architecture|<font color="#069" face="">'''System Architecture'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Testing|<font color="#069" face="">'''Testing'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_UI_Prototypes|<font color="#069" face="">'''UI Prototypes'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Knowledge_Base|<font color="#069" face="">'''Knowledge Base'''</font>]]
 +
*[[IS480_Team_wiki:_2013T2_Zora_Deliverables|<font color="#069" face="">'''Deliverables'''</font>]]
 +
|}
 +
 
 +
==Test Methodology==
 +
[[Image:Zora_test_methodology.png|600px]]<br>
 +
<br>
 +
 
 +
[[Image:Zora_Testing6.JPG|600px]]<br>
 +
 
 +
==Test Plan==
 +
[[Image:Zora_test_plan.png|600px]]<br>
 +
 
 +
==Test Cases==
 +
[[Image:Zora_Testing14.JPG|1000px]]<br>
 +
[[Media:Login_&_Logout.xlsx‎|Click here to view Login/Logout Test Case file.]]<br>
 +
[[Media:Remuneration.xlsx‎‎|Click here to view Remuneration Test Case file.]]<br>
 +
[[Media:Claim.xlsx‎‎|Click here to view Claim Test Case file.]]<br>
 +
[[Media:Leave.xlsx‎‎|Click here to view Leave Test Case file.]]<br>
 +
[[Media:Training Development.xlsx‎‎|Click here to view Training & Development Test Case file.]]<br>
 +
[[Media:Performance Management & Apprasial.xlsx‎‎|Click here to view Performance Management & Appraisal Test Case file.]]<br>
 +
[[Media:Administration.xlsx‎‎|Click here to view Administration Test Case file.]]<br>
 +
<br>
 +
 
 +
==Test Outcome==
 +
===Heuristic Evaluation 1===
 +
<b>Testers are the expert evaluators. They used the following heuristic and severity ratings, as shown below, to evaluate our application</b><br>
 +
{| class="wikitable centered"
 +
!Heuristics
 +
!Severity Ratings
 +
|-
 +
|style="vertical-align:text-top"|
 +
 
 +
1. Visibility of system status
 +
 
 +
2. Match between system and the real world
 +
 
 +
3. User control and freedom
 +
 
 +
4. Consistency and standards
 +
 
 +
5. Error prevention
 +
 
 +
6. Recognition rather than recall
 +
 
 +
7. Flexibility and efficiency of use
 +
 
 +
8. Aesthetic and minimalist design
 +
 
 +
9. Help users recognize, diagnose, and recover from errors
 +
 
 +
10. Help and documentation
 +
 
 +
Not related to any heuristic
 +
 
 +
|style="vertical-align:text-top"|
 +
 
 +
0 = I don't agree that this is a usability problem
 +
 
 +
1 = Cosmetic problem only
 +
 
 +
2 = Minor usability problem
 +
 
 +
3 = Major usability problem
 +
 
 +
4 = Usability catastrophe
 +
 
 +
|-
 +
|}
 +
<b>Images taken during the evaluation</b>
 +
{|- valign="top"
 +
| [[Image:Zora_Testing16.JPG‎ ‎‎|thumb|360px|left|Yun Xi bringing Hui Shia through the remuneration part of the project]]||[[Image:Zora_Testing17.JPG‎ ‎‎|thumb|360px|left|Mei Zhen explaining our prototype to Jing Yi]] || [[Image:Zora_Testing18.JPG‎ ‎‎|thumb|280px|left|Yun Xi going through the prototype with Eng Sen]]
 +
|}
 +
{|- valign="top"
 +
| [[Image:Zora_Testing19.JPG‎ ‎‎|thumb|240px|left|Yi Xian going through our prototype to Ken Khoon]]
 +
|}
 +
<b>Results obtained from the evaluation conducted</b>
 +
[[Image:Zora_Testing20.JPG‎ ‎‎|thumb|500px|left]]
 +
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
 +
 
 +
===User Test 1===
 +
<b>Steps taken to conduct user testing 1</b>
 +
<br>
 +
<b>Post-Test</b><br>
 +
Identify:
 +
# Goals of the test
 +
# Scope of the test
 +
# Participant Profile<br>
 +
Decides on: <br>
 +
# Testing set up (venue, time etc.)<br>
 +
Prepare:<br>
 +
# Testing items and documents<br>
 +
Recruit Testers<br>
 +
# Testers to read the study script<br>
 +
<br>
 +
 
 +
<b>During-Test</b>
 +
# Give user informed form to testers
 +
# Give task instructions to testers
 +
# Give interview form to testers
 +
# Observe and fill up Observation Form
 +
# Give Questionnaire to testers to fill up while testing<br>
 +
<br>
 +
 
 +
<b>Post-Test</b>
 +
# Consolidate Feedbacks
 +
# Analyze Results
 +
# Maintain a Response Plan
 +
# Implement feedback in UT1 and test again in upcoming UT2
 +
<br>
 +
 
 +
<b>Results obtained from the testing conducted</b>
 +
The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:
 +
 
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Data'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Pilot Participant'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 1'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 2'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 3'''
 +
 
 +
|-
 +
|
 +
| <center>[[Image:IMG 4600.JPG | 250px]]</center>
 +
| <center>[[Image:Zora_Testing22.JPG‎ | 150px]]</center>
 +
| <center>[[Image:Zora_Testing23.JPG‎ | 200px]]</center>
 +
| <center>[[Image:Zora Testing24.JPG | 240px]]</center>
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | General Data
 +
|-
 +
 
 +
|-
 +
|Age
 +
|35
 +
|50
 +
|23
 +
|26
 +
 
 +
|-
 +
|Occupation
 +
|Business Development Manager
 +
|Administrator
 +
|Student
 +
|Accountant
 +
 
 +
|-
 +
|Test Duration
 +
|30 mins
 +
|2 hrs
 +
|33 mins
 +
|29 mins
 +
 
 +
|-
 +
|Browser
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
 
 +
|-
 +
|Experienced in HRMS (Y/N)
 +
|Y
 +
|Y
 +
|N
 +
|N
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Remuneration (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to generate all payment slip
 +
|25 seconds
 +
|50 seconds
 +
|15 seconds
 +
|25 seconds
 +
 
 +
|-
 +
|Time taken to update payment status
 +
|10 seconds
 +
|1 minute
 +
|15 seconds
 +
|10 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|3
 +
|5
 +
|5
 +
|5
 +
 
 +
|-
 +
|Total number of question asked
 +
|1
 +
|3
 +
|2
 +
|2
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Claims (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to submit new claims
 +
|2 minutes
 +
|1 minute
 +
|1 minute
 +
|30 seconds
 +
 
 +
|-
 +
|Time taken to validate submitted claims
 +
|1 minute
 +
|20 seconds
 +
|30 seconds
 +
|20 seconds
 +
 
 +
|-
 +
|Time taken to approve/ reject submitted claims
 +
|10 seconds
 +
|10 seconds
 +
|5 seconds
 +
|10 seconds
 +
 
 +
|-
 +
|Time taken to view claim status
 +
|5 seconds
 +
|10 seconds
 +
|5 seconds
 +
|6 seconds
 +
 
 +
|-
 +
|Time taken to generate claim slip
 +
|5 seconds
 +
|10 seconds
 +
|10 seconds
 +
|25 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|17
 +
|16
 +
|18
 +
|17
 +
 
 +
|-
 +
|Total number of question asked
 +
|5
 +
|3
 +
|1
 +
|7
 +
 
 +
|-
 +
|Total number of bug found
 +
|2
 +
|0
 +
|1
 +
|1
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Leave(Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to apply leave
 +
|2 minutes
 +
|1 minute
 +
|1 minute
 +
|25 seconds
 +
 
 +
|-
 +
|Time taken to edit leave
 +
|30 seconds
 +
|40 seconds
 +
|1 minute
 +
|35 seconds
 +
 
 +
|-
 +
|Time taken to approve/reject leave
 +
|20 seconds
 +
|30 seconds
 +
|30 seconds
 +
|10 seconds
 +
 
 +
|-
 +
|Time taken to view leave that are being approved
 +
|10 seconds
 +
|30 seconds
 +
|10 seconds
 +
|5 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|10
 +
|11
 +
|12
 +
|11
 +
 
 +
|-
 +
|Total number of question asked
 +
|4
 +
|1
 +
|2
 +
|3
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
|}
 +
 
 +
 
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Category'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Pilot Participant'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 1'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 2'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 3'''
 +
 
 +
|-
 +
|Navigation/Login/Overall look
 +
|
 +
*Ugly layout for navigation bar
 +
|
 +
*Poor navigation bar where links are hard to find
 +
*Suggest to have vertical links instead
 +
*Suggest for navigation bar links to be clicked instead of hover
 +
|
 +
*Mouse-over very sensitive for the navigation bar
 +
*Navigation bar too long
 +
|
 +
*Layout is not very nice
 +
 
 +
|-
 +
|Remuneration
 +
|
 +
*Not a need to have prompt message for update payment status
 +
|
 +
*Suggest to have the Month the payslip is on to be at the top of the payslip
 +
*Suggest to include basic pay in the pay slip
 +
|
 +
*Payslip description is a bit vague
 +
*Include basic pay in pay slip
 +
|
 +
*Basic salary not shown in payslip
 +
 
 +
|-
 +
|Claims
 +
|
 +
*Upload button is redundant
 +
|
 +
*Unclear table headings
 +
*Unable to view claim submitted
 +
*Incorrect naming of button of "Generate Pay Slip", it should be "Generate Claim Slip"
 +
|
 +
*Add or remove row button inserted at the bottom of table
 +
*Button is wrongly named. Should be “Generate claim slip” instead
 +
|
 +
*Adding new row to submit another claim is not clear
 +
*The value for the button is wrong. It should be Generate Claim slip
 +
 
 +
|-
 +
|Leave
 +
|
 +
*Cannot find arrow on selection on calendar
 +
*Inconsistent spelling
 +
|
 +
*System should calculate the number of days taken and make it more obvious
 +
*Suggest "Confirm" button to calculate balance if leave is approved
 +
*Radio button look like bullet point
 +
|
 +
*Radio button should be white when unclicked
 +
*Page is very clustered with a lot of information
 +
*Button size not standardize
 +
|
 +
*Calendar is not working properly
 +
*Too many things cramped into one page
 +
*Naming not good
 +
*Button size not standardize
 +
 
 +
|-
 +
|}
 +
 
 +
===User Test 2===
 +
 
 +
<b>Results obtained from the testing conducted</b>
 +
The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:
 +
 
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Data'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Pilot Participant'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 1'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 2'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 3'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 4'''
 +
|-
 +
|
 +
| <center>[[Image:ZORA UT2 PP.JPG | 150px]]</center>
 +
| <center>[[Image:ZORA UT2 Ang.JPG‎ | 150px]]</center>
 +
| <center>[[Image:ZORA UT2 LC.JPG | 150px]]</center>
 +
| <center>[[Image:ZORA UT2 Ro.JPG | 150px]]</center>
 +
| <center>[[Image:ZORA UT2 FR.JPG | 150px]]</center>
 +
 
 +
|-
 +
! colspan="6" style="background: #A9F5E1;" | General Data
 +
|-
 +
 
 +
|-
 +
|Age
 +
|35
 +
|45
 +
|36
 +
|31
 +
|35
 +
 
 +
|-
 +
|Occupation
 +
|Business Development Manager
 +
|HR Personnel
 +
|Contract Manager (HM)
 +
|Structural Draftsman
 +
|Quantity Surveyor
 +
 
 +
|-
 +
|Test Duration
 +
|1 hour 30 mins
 +
|1 hour 40 mins
 +
|1 hour 10 mins
 +
|1 hour 20 mins
 +
|1 hour 25 mins
 +
 
 +
|-
 +
|Browser
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
 
 +
|-
 +
|Experienced in HRMS (Y/N)
 +
|Y
 +
|N
 +
|N
 +
|N
 +
|N
 +
 
 +
|-
 +
! colspan="6" style="background: #A9F5E1;" | Claims (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to submit new claims
 +
|59 seconds
 +
|2 minutes 36 seconds
 +
|2 minutes 30 seconds
 +
|1 minute 45 seconds
 +
|1 minute 35 seconds
 +
 
 +
|-
 +
|Time taken to validate submitted claims
 +
|1 minute 27 seconds
 +
|2 minutes 9 seconds
 +
|1 minute 40 seconds
 +
|2 minutes 20 seconds
 +
|2 minutes 10 seconds
 +
 
 +
|-
 +
|Time taken to edit payment method
 +
|41 seconds
 +
|1 minutes 18 seconds
 +
|24 seconds
 +
|45 seconds
 +
|36 seconds
 +
 
 +
|-
 +
|Time taken to approve/ reject submitted claims
 +
|56 seconds
 +
|3 minutes 29 seconds
 +
|2 minutes 6 seconds
 +
|1 minute 30 seconds
 +
|2 minutes 20 second
 +
 
 +
|-
 +
|Time taken to generate claim slip
 +
|26 seconds
 +
|N/A
 +
|N/A
 +
|N/A
 +
|N/A
 +
 
 +
|-
 +
|Time taken to view claim history
 +
|33 seconds
 +
|N/A
 +
|N/A
 +
|N/A
 +
|N/A
 +
 
 +
|-
 +
|Time taken to view claim status
 +
|12 seconds
 +
|21 seconds
 +
|22 seconds
 +
|25 seconds
 +
|28 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|22
 +
|25
 +
|28
 +
|30
 +
|34
 +
 
 +
|-
 +
|Total number of question asked
 +
|5
 +
|9
 +
|3
 +
|6
 +
|10
 +
 
 +
|-
 +
|Total number of bug found
 +
|4
 +
|0
 +
|0
 +
|1
 +
|0
 +
 
 +
|-
 +
! colspan="6" style="background: #A9F5E1;" | Leave (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to apply annual leave
 +
|33 seconds
 +
|2 minutes 16 seconds
 +
|1 minute 1 second
 +
|1 minute 15 seconds
 +
|1 minute 40 seconds
 +
 
 +
|-
 +
|Time taken to edit leave
 +
|33 seconds
 +
|1 minute 32 seconds
 +
|22 seconds
 +
|35 seconds
 +
|40 seconds
 +
 
 +
|-
 +
|Time taken to apply other leave
 +
|N/A
 +
|5 minutes 8 seconds
 +
|1 minute 45 seconds
 +
|1 minute 35 seconds
 +
|2 minutes
 +
 
 +
|-
 +
|Time taken to approve/reject leave
 +
|1 minute 27 seconds
 +
|3 minutes 18 seconds
 +
|1 minute 23 seconds
 +
|1 minute
 +
|1 minute 50 seconds
 +
 
 +
|-
 +
|Time taken to view status of leave & cancel leave
 +
|20 seconds
 +
|59 seconds
 +
|51 seconds
 +
|1 minute 15 seconds
 +
|3 minutes
 +
 
 +
|-
 +
|Time taken to view leave history
 +
|20 seconds
 +
|33 seconds
 +
|11 seconds
 +
|20 seconds
 +
|15 seconds
 +
 
 +
|-
 +
|Time taken to record medical leave
 +
|1 minute 30 seconds
 +
|3 minutes 30 seconds
 +
|2 minutes 6 seconds
 +
|2 minutes 20 seconds
 +
|2 minutes 10 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|64
 +
|65
 +
|58
 +
|62
 +
|60
 +
 
 +
|-
 +
|Total number of question asked
 +
|4
 +
|11
 +
|1
 +
|6
 +
|10
 +
 
 +
|-
 +
|Total number of bug found
 +
|1
 +
|1
 +
|1
 +
|1
 +
|0
 +
 
 +
|-
 +
! colspan="6" style="background: #A9F5E1;" | Remuneration (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to view past payment record for employees
 +
|N/A
 +
|22 seconds
 +
|10 seconds
 +
|18 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to view past payment record for workers
 +
|4 seconds
 +
|14 seconds
 +
|5 seconds
 +
|10 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to view past payment record for drivers
 +
|1 seconds
 +
|7 seconds
 +
|24 seconds
 +
|9 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to view past payment record for workers
 +
|N/A
 +
|22 seconds
 +
|51 seconds
 +
|35 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to generate all payment record for employees
 +
|N/A
 +
|22 seconds
 +
|51 seconds
 +
|35 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to generate all payment record for workers
 +
|N/A
 +
|26 seconds
 +
|7 seconds
 +
|15 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to generate all payment record for drivers
 +
|N/A
 +
|15 seconds
 +
|12 seconds
 +
|10 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to view personal payment record
 +
|N/A
 +
|36 seconds
 +
|27 seconds
 +
|24 seconds
 +
|N/A
 +
 
 +
|-
 +
|Time taken to input timesheet for drivers
 +
|2 minutes 14 seconds
 +
|1 minute 31 seconds
 +
|1 minute 9 seconds
 +
|2 minute
 +
|N/A
 +
 
 +
|-
 +
|Time taken to input timesheet for workers
 +
|N/A
 +
|3 minute 5 seconds
 +
|2 minute 19 seconds
 +
|2 minute 30 seconds
 +
|N/A
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|60
 +
|65
 +
|60
 +
|66
 +
|N/A
 +
 
 +
|-
 +
|Total number of question asked
 +
|1
 +
|1
 +
|0
 +
|1
 +
|N/A
 +
 
 +
|-
 +
|Total number of bug found
 +
|1
 +
|5
 +
|5
 +
|1
 +
|N/A
 +
 
 +
|-
 +
! colspan="6" style="background: #A9F5E1;" | Performance Management & Appraisal (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to create employees appraisal
 +
|45 seconds
 +
|2 minutes 36 seconds
 +
|1 minute 47 seconds
 +
|2 minutes
 +
|1 minute 30 seconds
 +
 
 +
|-
 +
|Time taken to approve/revise salary increment requests
 +
|23 seconds
 +
|26 seconds
 +
|29 seconds
 +
|1 minute 45 seconds
 +
|1 minute 15 seconds
 +
 
 +
|-
 +
|Time taken to distribute bonus
 +
|50 seconds
 +
|2 minutes 38 seconds
 +
|2 minutes 24 seconds
 +
|50 seconds
 +
|45 seconds
 +
 
 +
|-
 +
|Time taken to view scheduled increment/bonus
 +
|N/A
 +
|19 seconds
 +
|8 seconds
 +
|10 seconds
 +
|12 seconds
 +
 
 +
|-
 +
|Time taken to view increment history
 +
|N/A
 +
|24 seconds
 +
|12 seconds
 +
|10 seconds
 +
|12 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|43
 +
|40
 +
|45
 +
|41
 +
|39
 +
 
 +
|-
 +
|Total number of question asked
 +
|2
 +
|7
 +
|3
 +
|3
 +
|3
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="6" style="background: #A9F5E1;" | System Administration (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to create new profile
 +
|4 minutes 3 seconds
 +
|7 minutes 37 seconds
 +
|3 minutes 53 seconds
 +
|6 minutes
 +
|5 minutes 20 seconds
 +
 
 +
|-
 +
|Time taken to update profile
 +
|2 minutes 12 seconds
 +
|2 minutes 26 seconds
 +
|1 minute 30 seconds
 +
|1 minute
 +
|34 seconds
 +
 
 +
|-
 +
|Time taken to manage default number of leave
 +
|35 seconds
 +
|1 minute 40 seconds
 +
|1 minute 20 seconds
 +
|45 seconds
 +
|35 seconds
 +
 
 +
|-
 +
|Total number of clicks to finish this module
 +
|37
 +
|40
 +
|38
 +
|36
 +
|35
 +
 
 +
|-
 +
|Total number of question asked
 +
|1
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
|Total number of bug found
 +
|1
 +
|1
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
|}
 +
 
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 15%; background:#66CCCC; color:black;"|'''Category'''
 +
!style = "width: 17%; background:#66CCCC; color:black;"|'''Pilot Participant'''
 +
!style = "width: 17%; background:#66CCCC; color:black;"|'''Participant 1'''
 +
!style = "width: 17%; background:#66CCCC; color:black;"|'''Participant 2'''
 +
!style = "width: 17%; background:#66CCCC; color:black;"|'''Participant 3'''
 +
!style = "width: 17%; background:#66CCCC; color:black;"|'''Participant 4'''
 +
 
 +
|-
 +
|Navigation/Login/Overall look
 +
|
 +
*Fonts need to be bigger
 +
*User-friendly with nothing too overwhelming
 +
*Description and titles are simple and not too wordy
 +
|
 +
*Find it difficult to access the pages
 +
|
 +
*Generally fluid with navigation
 +
|
 +
*Make the date format in a format everyone is used to (I.e. dd/mm/yyyy)
 +
|
 +
*Application is helpful and easy to use
 +
 
 +
|-
 +
|Claim
 +
|
 +
*N/A
 +
|
 +
*Had difficulty finding the receipt for viewing
 +
*Did not know the need to hit "Update" after editing the amount
 +
|
 +
*Hard to locate the claim receipt
 +
|
 +
*N/A
 +
|
 +
*N/A
 +
 
 +
|-
 +
|Leave
 +
|
 +
*N/A
 +
|
 +
*Did not notice the dropdown in Record Leave
 +
|
 +
*Did not know how to upload more than 1 file
 +
|
 +
*N/A
 +
|
 +
*N/A
 +
 
 +
|-
 +
|Remuneration
 +
|
 +
*PDF can have more space
 +
|
 +
*N/A
 +
|
 +
*N/A
 +
|
 +
*Minus sign in the pay slip for those value that is going to be deducted
 +
*Add date for every signature
 +
|
 +
*N/A
 +
 
 +
|-
 +
|Performance & Appraisal
 +
|
 +
*N/A
 +
|
 +
*N/A
 +
|
 +
*N/A
 +
|
 +
*Approve does not automatically save the new value, and also allow to revise the value just in case the value is wrong
 +
*The title for view scheduled increments/bonus seemed confusing
 +
|
 +
*N/A
 +
 
 +
|-
 +
|System Administration
 +
|
 +
*N/A
 +
|
 +
*Colour of username is available should not be red in color
 +
|
 +
*Ajax is not working in Firefox
 +
*Have a dropdown to select department
 +
*Have validation for phone numbers and postal code
 +
|
 +
*N/A
 +
|
 +
*N/A
 +
 
 +
|-
 +
|}
 +
 
 +
<h3>Response Plan (Implement)</h3>
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 10%; background:#66CCCC; color:black;"|'''S/N'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Functionality'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Feedback'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Solution'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Decision'''
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Claim
 +
|-
 +
 
 +
|-
 +
|1
 +
|Submit Claim
 +
|User tend to click "Add Claim" instead of "Submit Claim"
 +
|Will shift the "Add Claim" button to somewhere else"
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|validate Claim & Approve/Reject Claim
 +
|User cannot find where to view the claim receipt
 +
|Make the "view" link bigger and more distinct
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Leave
 +
|-
 +
 
 +
|-
 +
|1
 +
|Apply Leave
 +
|User is unsure of how to upload more than 1 files while applying for leave that require supporting documents
 +
|Have a short instruction before the upload file field (i.e. Hold down CTRL button to select multiple options)
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|Apply Leave
 +
|User is unsure of what to put in the "Date To" field to apply leave on the same day
 +
|When user selects the date in "Date From" field, the same date should be repopulated in the "Date To" field
 +
|Implement
 +
 
 +
|-
 +
|3
 +
|Apply Leave/Record Leave
 +
|User tends to type the date instead of choosing the date using the datepicker leading to the wrong format being input
 +
|Have a short instruction before the "Date To" and "Date From" field to indicate the date format and give an example of it
 +
|Implement
 +
 
 +
|-
 +
|4
 +
|User should be able to view leave of their colleagues in the same department so that they can plan how to cover up leave
 +
|Have a separate "View Leave" page for users to view leave of people from the same department
 +
|Make the "view" link bigger and more distinct
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Remuneration
 +
|-
 +
 
 +
|-
 +
|1
 +
|View Personal Payslip tab
 +
|User feel that it is hard to navigate to view personal payment record link when it is at the bottom
 +
|Change the arrangement for the links under remuneration tab
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|View payslip
 +
|User finds that the less part of the payslip can be more obvious with a "-" sign
 +
|Include the "-" sign in the PDF payslip
 +
|Implement
 +
 
 +
|-
 +
|3
 +
|View payslip
 +
|Instead of month only, should include year as well in the payslip
 +
|Include the year
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Performance & Appraisal
 +
|-
 +
 
 +
|-
 +
|1
 +
|Edit Salary Increment
 +
|User does not click the "Save" button when revising the salary increment amont. As a result, the amount approved is not the revised amount. It is not obvious that the "Save" button should be clicked before submitting the form
 +
|In the event the increment value is not saved, the user should not be allowed to approve the increment request
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|View Scheduled Salary Increment/Scheduled Bonus
 +
|Currently the "Scheduled salary increment" and "Scheduled Bonuses" tables reside on the same page ("View Scheduled Increments/Bonus"). This can be confusing to the user as he/she is unable to distinguish between scheduled salary increments and scheduled bonuses
 +
|Change the colours of the tables to differentiate between scheduled salary increments and scheduled bonuses
 +
|Implement
 +
 
 +
|-
 +
|3
 +
|Submit Appraisal
 +
|Placement of "Submit Appraisal" and "Clear" buttons. Button placements are different from other modules. Not intuitive and users may get confused
 +
|Standardize position of buttons across modules
 +
|Implement
 +
 
 +
|-
 +
|4
 +
|Submit Appraisal
 +
|When an appaisal has been submitted successfully, it is not obvious to the user
 +
|Display a successful appraisal submission message
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | System Administration
 +
|-
 +
 
 +
|-
 +
|1
 +
|Manage Leave
 +
|Change default no. of leave --> User unsure of change is successful cos there is no popup or anything
 +
|Display a successful update message
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|Create/Edit Profile
 +
|Remind the user the DOB format to type. User unsure of what to type. The date thing also must select super long ago. super troublesome for them
 +
|Display a date format beside the field
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Navigation
 +
|-
 +
 
 +
|-
 +
|1
 +
|Navigation
 +
|In the navigation bar, users tend to click on menu items which display sub-menus, this results in the menu options being minimize. Selection of sub-menu items require users to hover over the main menu items and then clicking the sub-menus items
 +
|Enable the navigation bar to support both hover and click functionality. In both instances, the sub-menu will be displayed and the menu will not be minimized
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Login
 +
|-
 +
 
 +
|-
 +
|1
 +
|Error message
 +
|Error message for login is black in colour
 +
|Error message for login should be red in colour
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|Active status
 +
|Account with inactive status can login
 +
|Account with inactive status should not be able login
 +
|Implement
 +
|-
 +
|}
 +
 
 +
{| cellpadding="0" style="margin: 1em auto 1em auto;" "height: 317px"  width="750px"
 +
! scope="col" width="200" bgcolor="#a7b1aa"|
 +
! scope="col" width="550" bgcolor="#a7b1aa"|<font color="white">Changes made from User Test 2 Feedbacks</font>
 +
|- align="left"
 +
|bgcolor="#cedad1"|Claims
 +
|bgcolor="#cedad1"|
 +
[[Image:zora_claim_1.jpg|550|center]]
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Leave
 +
|bgcolor="#cedad1"|
 +
[[Image:zora_leave_1.jpg|550|center]]
 +
|
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Remuneration
 +
|bgcolor="#cedad1"|
 +
[[Image:zora_rem_1.jpg|550|center]]
 +
|
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Performance Management & Appraisal
 +
|bgcolor="#cedad1"|
 +
[[Image:zora_pma_1.jpg|550|center]]
 +
|
 +
|}
 +
 
 +
{| cellpadding="0" style="margin: 1em auto 1em auto;" "height: 317px"  width="750px"
 +
! scope="col" width="200" bgcolor="#a7b1aa"|
 +
! scope="col" width="550" bgcolor="#a7b1aa"|<font color="white">Quantitative Feedback Analysis</font>
 +
|- align="left"
 +
|bgcolor="#cedad1"|Ease of Use
 +
|bgcolor="#cedad1"|
 +
[[Image:zora_quant_1.jpg|350|center]]
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Overall Layout
 +
|bgcolor="#cedad1"|
 +
[[Image:zora_quant_2.jpg|350|center]]
 +
|
 +
|}
 +
 
 +
{| align="center" cellpadding="2" style=  width="750px"
 +
|- align="left"
 +
! scope="col"  bgcolor="#4D4D4D"| <font color="white">User Test 2 Analysis</font>
 +
|- align="left"
 +
! scope="col"  bgcolor="#ffffff"| [[Media:User Test 2 Analysis.pdf|<font color="#069" face="">'''
 +
View Full Analysis. '''</font>]]
 +
|}
 +
 
 +
<h3>Response Plan (Not Implementing)</h3>
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 10%; background:#66CCCC; color:black;"|'''S/N'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Functionality'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Feedback'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Reason for Not Implementing'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Decision'''
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Leave
 +
|-
 +
 
 +
|-
 +
|1
 +
|Approve/Reject Leave
 +
|People in the department should have the authority of approving/rejecting the leave the user sends
 +
|The sponsor of the project mentioned that the line manager should be the only one approving. He should know if anyone in the department rejects to that particular leave on a particular day in case of any project that arises. Hence, according to the business logic, only the line manager should have the authority to approve/reject leave
 +
|Not Implementing
 +
 
 +
|-
 +
|2
 +
|Apply Leave
 +
|User takes quite a while to find the leave balance on the "Apply Leave" page
 +
|As the task instruction explicitly mention to the tester to check on the leave balance, tester might not know that it can be checked in the "Apply Leave" page. Moreover, it is only 1 out of 5 of the user that mentioned that. However, everyone eventually managed to find the leave balance when they are applying leave
 +
|Not Implementing
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Remuneration
 +
|-
 +
 
 +
|-
 +
|1
 +
|Payslip
 +
|Suggestion: For payslip, put signature and date
 +
|The payslip is followed closely to the current payslip that the company is using
 +
|Not Implementing
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Performance & Appraisal
 +
|-
 +
 
 +
|-
 +
|1
 +
|Edit Appraisal
 +
|User should have the option to edit Appraisal once submitted
 +
|Client explicitly mentioned that Appraisals should not be edited once submitted
 +
|Not Implementing
 +
 
 
|-
 
|-
! [[IS480_Team_wiki:_2013T2_Zora|<font color="#069" face="">'''Home'''</font>]] !! Project Overview !! Project Management !!Documentation
+
! colspan="5" style="background: #A9F5E1;" | System Administration
 
|-
 
|-
| [[IS480_Team_wiki:_2013T2_Zora#Notifications|<font color="#069" face="">'''Notifications'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_Project_Description|<font color="#069" face="">'''Project Description'''</font>]] || [[IS480_Team_wiki:_2013T2_Zora_Project_Schedule|<font color="#069" face="">'''Project Schedule'''</font>]] || [[IS480_Team_wiki:_2013T2_Zora_Meeting_Minutes|<font color="#069" face="">'''Meeting Minutes'''</font>]]
+
 
 
|-
 
|-
| [[IS480_Team_wiki:_2013T2_Zora#Progress|<font color="#069" face="">'''Progress'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_Stakeholders|<font color="#069" face="">'''Stakeholders'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_Framework|<font color="#069" face="">'''Framework'''</font>]] || [[IS480_Team_wiki:_2013T2_Zora_Business_Requirements|<font color="#069" face="">'''Business Requirements'''</font>]]
+
|1
 +
|Create/Edit Profile
 +
|Creating user profile --> User tend to click Tab. Take note to rearrange input data values. E.g. Put "Given Name" beside "Family Name" cos that is where the cursor go when user click Tab. Do take note of this
 +
|Not really important
 +
|Not Implementing
 +
 
 
|-
 
|-
| [[IS480_Team_wiki:_2013T2_Zora#Milestones|<font color="#069" face="">'''Milestones'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_X_Factor|<font color="#069" face="">'''X Factor'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_Scope|<font color="#069" face="">'''Scope'''</font>]] ||[[IS480_Team_wiki:_2013T2_Zora_System_Architecture|<font color="#069" face="">'''System Architecture'''</font>]]
+
|2
 +
|Create/Edit Profile
 +
|To input allowance --> amount for which allowance type? If user select more than 1 allowance type, the $500 is for which one?
 +
|It suppose to be like this, is a combine amount
 +
|Not Implementing
 +
 
 
|-
 
|-
| || [[IS480_Team_wiki:_2013T2_Zora_Team_Zora|<font color="#069" face="">'''Team Zora'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_Metrics|<font color="#069" face="">'''Metrics'''</font>]]||[[IS480_Team_wiki:_2013T2_Zora_Testing|<font color="#069" face="">'''Testing'''</font>]]
+
! colspan="5" style="background: #A9F5E1;" | Login/Logout
 
|-
 
|-
| || [[IS480_Team_wiki:_2013T2_Zora_Learning_Outcomes|<font color="#069" face="">'''Learning Outcomes'''</font>]]|| [[IS480_Team_wiki:_2013T2_Zora_Risks|<font color="#069" face="">'''Risks'''</font>]] || [[IS480_Team_wiki:_2013T2_Zora_UI_Prototypes|<font color="#069" face="">'''UI Prototypes'''</font>]]
+
 
 
|-
 
|-
| || || || [[IS480_Team_wiki:_2013T2_Zora_Knowledge_Base|<font color="#069" face="">'''Knowledge Base'''</font>]]
+
|1
 +
|Logout
 +
|User takes a few secs to find logout button
 +
|Attributed to lack of familiarity with the system user interface.
 +
|Not Implementing
 +
 
 +
|-
 +
|}
 +
 
 +
===User Test 3===
 +
 
 +
In this user test, we did the testing on 9 participants, of which 5 participants are the same person from user test 2. This is so that we are able to do comparison between the qualitative and quantitative results gathered from user test 2 with the results that we gathered in this user test 3. Due to the space constraints, we will be showing the results gathered from the 4 new participants. If you want to view more results from user test 3, please click on this -> [[Media:User Test 3 Analysis.pdf|<font color="#069" face="">'''
 +
View Full Analysis. '''</font>]]
 +
 
 +
<b>Results obtained from the testing conducted</b>
 +
The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:
 +
 
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Data'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 1'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 2'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 3'''
 +
!style = "width: 20%; background:#66CCCC; color:black;"|'''Participant 4'''
 +
 
 +
|-
 +
|
 +
| <center>[[Image:Zora UT3 Participant 6.png | 200px]]</center>
 +
| <center>[[Image:ZORA UT3 CHING.JPG | 200px]]</center>
 +
| <center>[[Image:ZORA UT3 RAS.jpg | 200px]]</center>
 +
| <center>[[Image:ZORA UT3 STU.JPG | 200px]]</center>
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | General Data
 +
|-
 +
 
 +
|-
 +
|Age
 +
|28
 +
|30
 +
|30
 +
|20
 +
 
 +
|-
 +
|Occupation
 +
|Design Engineer
 +
|Quantity Surveyor
 +
|Work & Safety Health Officer
 +
|Student Intern
 +
 
 +
|-
 +
|Test Duration
 +
|1 hour 15 mins
 +
|1 hour
 +
|1 hour 4 mins
 +
|1 hour 30 mins
 +
 
 +
|-
 +
|Browser
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
|Google Chrome
 +
 
 +
|-
 +
|Experienced in HRMS (Y/N)
 +
|N
 +
|N
 +
|N
 +
|N
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Claims (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to submit new claims
 +
|1 minute 27 seconds
 +
|37 seconds
 +
|1 minute
 +
|1 minute 54 seconds
 +
 
 +
|-
 +
|Time taken to validate submitted claims
 +
|1 minute 21 seconds
 +
|1 minutes 14 seconds
 +
|2 minute 26 seconds
 +
|1 minutes 8 seconds
 +
 
 +
|-
 +
|Time taken to edit payment method
 +
|22 seconds
 +
|15 seconds
 +
|37 seconds
 +
|43 seconds
 +
 
 +
|-
 +
|Time taken to approve/ reject submitted claims
 +
|25 seconds
 +
|29 seconds
 +
|58 seconds
 +
|42 seconds
 +
 
 +
|-
 +
|Time taken to generate claim slip
 +
|2 minutes 32 seconds
 +
|59 seconds
 +
|2 minutes 56 seconds
 +
|40 seconds
 +
 
 +
|-
 +
|Time taken to view claim history
 +
|1 minute 14 seconds
 +
|30 seconds
 +
|41 seconds
 +
|20 seconds
 +
 
 +
|-
 +
|Time taken to view claim status
 +
|27 seconds
 +
|20 seconds
 +
|51 seconds
 +
|9 seconds
 +
 
 +
|-
 +
|Time taken to generate claim report
 +
|30 seconds
 +
|25 seconds
 +
|40 seconds
 +
|27 seconds
 +
 
 +
|-
 +
|Total number of question asked
 +
|0
 +
|4
 +
|7
 +
|0
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Leave (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to apply & edit annual leave
 +
|2 minutes 37 seconds
 +
|1 minutes 19 seconds
 +
|2 minute 36 second
 +
|1 minute 50 seconds
 +
 
 +
|-
 +
|Time taken to apply other leave
 +
|2 minutes 55 seconds
 +
|2 minutes
 +
|1 minute 44 seconds
 +
|1 minute
 +
 
 +
|-
 +
|Time taken to approve leave
 +
|2 minutes 36 seconds
 +
|37 seconds
 +
|3 minutes 15 seconds
 +
|2 minutes 10 seconds
 +
 
 +
|-
 +
|Time taken to view status of leave & cancel leave
 +
|1 minute 24 seconds
 +
|55 seconds
 +
|1 minute 20 seconds
 +
|2 minute 4 seconds
 +
 
 +
|-
 +
|Time taken to view leave history
 +
|10 seconds
 +
|10 seconds
 +
|1 minute 25 seconds
 +
|46 seconds
 +
 
 +
|-
 +
|Time taken to record medical leave
 +
|2 minutes 47 seconds
 +
|1 minute 44 seconds
 +
|1 minute 45 seconds
 +
|2 minutes 25 seconds
 +
 
 +
|-
 +
|Time taken to record unpaid leave & approve recorded leave
 +
|2 minutes 51 seconds
 +
|1 minutes 50 seconds
 +
|3 minutes 5 seconds
 +
|2 minutes 10 seconds
 +
 
 +
|-
 +
|Time taken to generate leave report
 +
|15 seconds
 +
|56 seconds
 +
|2 minutes  35 seconds
 +
|15 seconds
 +
 
 +
|-
 +
|Total number of question asked
 +
|1
 +
|2
 +
|6
 +
|3
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Remuneration (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to generate all payment record for employees
 +
|1 minute 12 seconds
 +
|18 seconds
 +
|45 seconds
 +
|52 seconds
 +
 
 +
|-
 +
|Time taken to view past payment record for employees
 +
|7 seconds
 +
|20 seconds
 +
|49 seconds
 +
|13 seconds
 +
 
 +
|-
 +
|Time taken to view personal payment record
 +
|33 seconds
 +
|16 seconds
 +
|55 seconds
 +
|20 seconds
 +
 
 +
|-
 +
|Time taken to input timesheet for workers
 +
|3 minutes 19 seconds
 +
|1 minutes 18 seconds
 +
|2 minutes 45 seconds
 +
|3 minutes 23 seconds
 +
 
 +
|-
 +
|Time taken to input OT timesheet for drivers
 +
|1 minute 18 seconds
 +
|23 seconds
 +
|1 minute 47 seconds
 +
|3 minute 3 seconds
 +
 
 +
|-
 +
|Time taken to view worker costing
 +
|13 seconds
 +
|5 seconds
 +
|1 minute 19 seconds
 +
|15 seconds
 +
 
 +
|-
 +
|Total number of question asked
 +
|2
 +
|0
 +
|4
 +
|3
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Performance Management & Appraisal (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to appraise employees
 +
|3 minutes 21 seconds
 +
|58 seconds
 +
|2 minute 21 seconds
 +
|3 minutes 19 seconds
 +
 
 +
|-
 +
|Time taken to distribute bonus
 +
|1 minute 11 seconds
 +
|30 seconds
 +
|1 minute 55 seconds
 +
|52 seconds
 +
 
 +
|-
 +
|Total number of question asked
 +
|0
 +
|0
 +
|4
 +
|1
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Training & Development (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to propose training
 +
|1 minutes 25 seconds
 +
|35 seconds
 +
|3 minutes
 +
|1 minutes 57 second
 +
 
 +
|-
 +
|Time taken to approve/reject training by HR
 +
|3 minutes 43 seconds
 +
|1 minute 55 seconds
 +
|6 minutes 3 seconds
 +
|2 minutes 55 seconds
 +
 
 +
|-
 +
|Time taken to approve/reject training by MD
 +
|49 seconds
 +
|30 seconds
 +
|1 minute 44 seconds
 +
|55 seconds
 +
 
 +
|-
 +
|Time taken to view proposed training status
 +
|26 seconds
 +
|14 seconds
 +
|47 seconds
 +
|1 minute 7 seconds
 +
 
 +
|-
 +
|Time taken to view nominated training
 +
|1 minute 3 seconds
 +
|38 seconds
 +
|1 minute 5 seconds
 +
|21 seconds
 +
 
 +
|-
 +
|Time taken to manage external training
 +
|1 minute 2 seconds
 +
|32 seconds
 +
|1 minute 35 seconds
 +
|1 minute 4 seconds
 +
 
 +
|-
 +
|Total number of question asked
 +
|0
 +
|0
 +
|5
 +
|0
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | System Administration (Quantitative Data Analysis)
 +
|-
 +
 
 +
|-
 +
|Time taken to manage default number of leave
 +
|1 minutes 41 seconds
 +
|24 seconds
 +
|1 minutes 22 seconds
 +
|31 seconds
 +
 
 +
|-
 +
|Time taken to create new profile
 +
|5 minutes 58 seconds
 +
|5 minutes 14 seconds
 +
|3 minute 55 seconds
 +
|4 minute 59 seconds
 +
 
 +
|-
 +
|Time taken to update profile
 +
|46 seconds
 +
|51 seconds
 +
|1 minute 7 seconds
 +
|1 minute 12 seconds
 +
 
 +
|-
 +
|Total number of question asked
 +
|0
 +
|0
 +
|5
 +
|1
 +
 
 +
|-
 +
|Total number of bug found
 +
|0
 +
|0
 +
|0
 +
|0
 +
 
 +
|-
 +
|}
 +
 
 +
<h3>Response Plan (Implement)</h3>
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 10%; background:#66CCCC; color:black;"|'''S/N'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Functionality'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Feedback'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Solution'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Decision'''
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Claim
 +
|-
 +
 
 +
|-
 +
|1
 +
|Submit Claim
 +
|Tedious to record receipt numbers of the claim receipts by HR
 +
|Have an addition field "Receipt No." for employee to key in their own receipt number and HR will just validate instead of having to input it herself
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|Validate Claim
 +
|Did not know need to update the edited amount before continuing to validate the claim
 +
|To include a "message" whenever the edit button is click to remind the user that update button must be hit before the changes can be made
 +
|Implement
 +
 
 +
|-
 +
|3
 +
|Approve/Reject Claim
 +
|Admin accept/reject the status not obvious
 +
|Make it bigger and more striking colour
 +
|Implement
 +
 
 +
|-
 +
|4
 +
|Edit Payment Method
 +
|Have a back button to generate claimslip
 +
|Implement "Go to Generate Claimslip" button below the edit payment method page
 +
|Implement
 +
 
 +
|-
 +
|5
 +
|Generate Claimslip
 +
|No need claimslip for employee
 +
|Take out the employee's copy of claimslip in the PDF generated
 +
|Implement
 +
 
 +
|-
 +
|6
 +
|Generate Claimslip
 +
|Make the slip bigger
 +
|Make the slip bigger
 +
|Implement
 +
 
 +
|-
 +
|7
 +
|Generate Claimslip
 +
|More space for signature
 +
|Spread the claim slip to have more space and include a date space to input date
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Leave
 +
|-
 +
 
 +
|-
 +
|1
 +
|Record Leave
 +
|Not obvious that employee's name is able to be selected by keying the alphabet
 +
|Enable filtering for user's to type employee's name
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|Emailing for all Leave
 +
|Leave approval emails did not CC HR personnel
 +
|CC HR personnel for all leave emails
 +
|Implement
 +
 
 +
|-
 +
|3
 +
|Approve/Reject Leave
 +
|Never state day duration in email (e.g. half day)
 +
|State day duration of leave taken in email
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Remuneration
 +
|-
 +
 
 +
|-
 +
|1
 +
|Input worker timesheet
 +
|Full name of workers should be shown for worker input timesheet page
 +
|Insert full name of workers
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Performance & Appraisal
 +
|-
 +
 
 +
|-
 +
|1
 +
|View full appraisal
 +
|Text on "View Appraisal" button is not specific
 +
|Change Button Text to "View Full Appraisal"
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|View full appraisal
 +
|Scores displayed are not meaningful to the user
 +
|Display of Remuneration Scores- show total (e.g. 100/100)
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Training & Development
 +
|-
 +
 
 +
|-
 +
|1
 +
|Approve/Reject Proposed Training
 +
|UI Changes for select the training to be approve/reject. Accept / Reject button unclear, user wasn't sure she needs to click the button to proceed
 +
|Change the name of the button
 +
|Implement
 +
 
 +
|-
 +
|2
 +
|Approve/Reject Proposed Training
 +
|UI Changes for the page that they input the training details. Back button not standardize. No need write "default". Venue nv put "*". Instead of add session, it should be add another training session. Different form of writing time (e.g. 9am vs 0900. Course duration give dropdown for hrs/mins
 +
|Standardize back button. Remove the default wording. Venue is a required. Change value for the button for adding training session. Put the indication that is a 24 hour (so accept numbers only). A dropdown box that have the hr/mins/day/month/year for course duration
 +
|Implement
 +
 
 +
|-
 +
|3
 +
|Propose Training
 +
|Need to check if the employee have or didn't have any people under him/her. Nomination of employee
 +
|Do a check if there is no employee under him, it will prompt him a message
 +
|Implement
 +
 
 +
|-
 +
|4
 +
|Manage External Training
 +
|Training Status from Completed change to Course Completed
 +
|Change from completed to Course Completed
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | System Administration
 +
|-
 +
 
 +
|-
 +
|1
 +
|Manage Employee
 +
|Adding the department dropdown list
 +
|Add a table in db + and add a dropdown list
 +
|Implement
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Navigation
 +
|-
 +
 
 +
|-
 +
|1
 +
|General
 +
|Font size could be larger
 +
|Increase Font Size
 +
|Implement
 +
|-
 +
|}
 +
 
 +
{| cellpadding="0" style="margin: 1em auto 1em auto;" "height: 317px"  width="750px"
 +
! scope="col" width="200" bgcolor="#a7b1aa"|
 +
! scope="col" width="550" bgcolor="#a7b1aa"|<font color="white">Changes made from User Test 3 Feedbacks</font>
 +
|- align="left"
 +
|bgcolor="#cedad1"|Claims
 +
|bgcolor="#cedad1"|
 +
<b>Before</b> [[Image:Zora UT3 CBefore.JPG|550|center]]
 +
<br/>
 +
<b>After</b> [[Image:Zora UT3 CAfter.JPG|550|center]]
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Leave
 +
|bgcolor="#cedad1"|
 +
<b>Before</b> [[Image:Zora UT3 LBefore.JPG|550|center]]
 +
<br/>
 +
<b>After</b> [[Image:Zora UT3 LAfter.JPG|550|center]]
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Training & Development
 +
|bgcolor="#cedad1"|
 +
<b>Before</b> [[Image:Zora UT3 TDBefore.JPG|550|center]]
 +
<br/>
 +
<b>After</b> [[Image:Zora UT3 TDAfter.JPG|550|center]]
 +
|
 +
|}
 +
 
 +
{| cellpadding="0" style="margin: 1em auto 1em auto;" "height: 317px"  width="750px"
 +
! scope="col" width="200" bgcolor="#a7b1aa"|
 +
! scope="col" width="550" bgcolor="#a7b1aa"|<font color="white">Quantitative Feedback - User Test 2 versus User Test 3</font>
 +
|- align="left"
 +
|bgcolor="#cedad1"|Ease of Use
 +
|bgcolor="#cedad1"|
 +
[[Image:Zora_Ease_of_use_analysis.png|400px|center]]
 +
<br/>
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Overall Layout
 +
|bgcolor="#cedad1"|
 +
[[Image:Zora_overall_layout_analysis.png|400px|center]]
 +
<br/>
 +
|
 +
|- align="left"
 +
|bgcolor="#cedad1"|Analysis for the same 5 participants from user test 2
 +
|bgcolor="#cedad1"|
 +
This is the analysis done from user test 2:
 +
[[Image:Zora_UT3_Before_Analysis.png|500px|center]]
 +
Reasons for the time taken:
 +
*Validate Submitted Claim - Most users are not able to locate the "view" claim receipt link
 +
*Edit Payment Method - Confirmation message was not display thus user did not know the payment method had been updated
 +
*Apply Other Leave - Most users was unsure how to upload 2 files as no instruction was stated hence a huge amount of time was spent figuring it out
 +
<br/>
 +
This is the analysis done for user test 3:
 +
[[Image:Zora_UT3_After_Analysis.png|500px|center]]
 +
Reasons for changes in timing:
 +
*Changes that we made and improved on the application was useful
 +
*Live Deployment + user interacting with the system before enables them to familiar with the system
 +
|
 +
|}
 +
 
 +
<h3>Response Plan (Not Implementing)</h3>
 +
{| class="wikitable centered" width="95%"
 +
!style = "width: 10%; background:#66CCCC; color:black;"|'''S/N'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Functionality'''
 +
!style = "width: 22%; background:#66CCCC; color:black;"|'''Feedback'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Reason for Not Implementing'''
 +
!style = "width: 23%; background:#66CCCC; color:black;"|'''Proposed Decision'''
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Leave
 +
|-
 +
 
 +
|-
 +
|1
 +
|Record medical leave
 +
|Did not check for duplicate medical series no
 +
|There is a possibility that 2 person can have the same medical series no. Hence, this will not be checked
 +
|Not Implementing
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Remuneration
 +
|-
 +
 
 +
|-
 +
|1
 +
|Worker payslip
 +
|Timesheet to include dinner breakdown (3h x 2) instead of giving the total ending sum
 +
|Dinner is allocated $3 for each dinner, if necessary. The payslip in the system is mirroring the current payslip they are using
 +
|Not Implementing
 +
 
 +
|-
 +
|2
 +
|Input worker timesheet
 +
|User tend to miss out the days but will notice when see days
 +
|The timesheet in the system is mirroring the current worker timesheet they used. Others may not be familiar but the HR personnel who is in charged of inputting the timesheet will be familiar if it is the same format as the one she is using
 +
|Not Implementing
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Performance & Appraisal
 +
|-
 +
 
 +
|-
 +
|1
 +
|Create Employee Appraisal
 +
|Radio button too small. Cannot click properly
 +
|Radio buttons are controlled by system and browser. Customization of radio buttons would not enable usage on mobile devices
 +
|Not Implementing
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | Training & Development
 +
|-
 +
 
 +
|-
 +
|1
 +
|Manage External Training
 +
|Suggest user can type "UOB" for cheque no. (so allow text and no.)
 +
|Cheque number got nothing to do with the Bank name
 +
|Not Implementing
 +
 
 +
|-
 +
! colspan="5" style="background: #A9F5E1;" | System Administration
 +
|-
 +
 
 +
|-
 +
|1
 +
|Manage Employee
 +
|Date Cannot go back to earlier than 2014. Wage Type, Hourly, Edit button on the top, cause too much to scroll down.
 +
|The requirement that we had agreed on earlier was to either put in daily wage or monthly wage
 +
|Not Implementing
 +
 
 
|-
 
|-
| || || || [[IS480_Team_wiki:_2013T2_Zora_Deliverables|<font color="#069" face="">'''Deliverables'''</font>]]
 
 
|}
 
|}
==Test Methodology==
 
==Test Plan==
 
==Test Cases==
 
==Test Results==
 

Latest revision as of 01:16, 13 April 2014

Zora page header.png
Home Project Overview Project Management Documentation

Test Methodology

Zora test methodology.png

Zora Testing6.JPG

Test Plan

Zora test plan.png

Test Cases

Zora Testing14.JPG
Click here to view Login/Logout Test Case file.
Click here to view Remuneration Test Case file.
Click here to view Claim Test Case file.
Click here to view Leave Test Case file.
Click here to view Training & Development Test Case file.
Click here to view Performance Management & Appraisal Test Case file.
Click here to view Administration Test Case file.

Test Outcome

Heuristic Evaluation 1

Testers are the expert evaluators. They used the following heuristic and severity ratings, as shown below, to evaluate our application

Heuristics Severity Ratings

1. Visibility of system status

2. Match between system and the real world

3. User control and freedom

4. Consistency and standards

5. Error prevention

6. Recognition rather than recall

7. Flexibility and efficiency of use

8. Aesthetic and minimalist design

9. Help users recognize, diagnose, and recover from errors

10. Help and documentation

Not related to any heuristic

0 = I don't agree that this is a usability problem

1 = Cosmetic problem only

2 = Minor usability problem

3 = Major usability problem

4 = Usability catastrophe

Images taken during the evaluation

Yun Xi bringing Hui Shia through the remuneration part of the project
Mei Zhen explaining our prototype to Jing Yi
Yun Xi going through the prototype with Eng Sen
Yi Xian going through our prototype to Ken Khoon

Results obtained from the evaluation conducted

Zora Testing20.JPG






















User Test 1

Steps taken to conduct user testing 1
Post-Test
Identify:

  1. Goals of the test
  2. Scope of the test
  3. Participant Profile

Decides on:

  1. Testing set up (venue, time etc.)

Prepare:

  1. Testing items and documents

Recruit Testers

  1. Testers to read the study script


During-Test

  1. Give user informed form to testers
  2. Give task instructions to testers
  3. Give interview form to testers
  4. Observe and fill up Observation Form
  5. Give Questionnaire to testers to fill up while testing


Post-Test

  1. Consolidate Feedbacks
  2. Analyze Results
  3. Maintain a Response Plan
  4. Implement feedback in UT1 and test again in upcoming UT2


Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Pilot Participant Participant 1 Participant 2 Participant 3
IMG 4600.JPG
Zora Testing22.JPG
Zora Testing23.JPG
Zora Testing24.JPG
General Data
Age 35 50 23 26
Occupation Business Development Manager Administrator Student Accountant
Test Duration 30 mins 2 hrs 33 mins 29 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) Y Y N N
Remuneration (Quantitative Data Analysis)
Time taken to generate all payment slip 25 seconds 50 seconds 15 seconds 25 seconds
Time taken to update payment status 10 seconds 1 minute 15 seconds 10 seconds
Total number of clicks to finish this module 3 5 5 5
Total number of question asked 1 3 2 2
Total number of bug found 0 0 0 0
Claims (Quantitative Data Analysis)
Time taken to submit new claims 2 minutes 1 minute 1 minute 30 seconds
Time taken to validate submitted claims 1 minute 20 seconds 30 seconds 20 seconds
Time taken to approve/ reject submitted claims 10 seconds 10 seconds 5 seconds 10 seconds
Time taken to view claim status 5 seconds 10 seconds 5 seconds 6 seconds
Time taken to generate claim slip 5 seconds 10 seconds 10 seconds 25 seconds
Total number of clicks to finish this module 17 16 18 17
Total number of question asked 5 3 1 7
Total number of bug found 2 0 1 1
Leave(Quantitative Data Analysis)
Time taken to apply leave 2 minutes 1 minute 1 minute 25 seconds
Time taken to edit leave 30 seconds 40 seconds 1 minute 35 seconds
Time taken to approve/reject leave 20 seconds 30 seconds 30 seconds 10 seconds
Time taken to view leave that are being approved 10 seconds 30 seconds 10 seconds 5 seconds
Total number of clicks to finish this module 10 11 12 11
Total number of question asked 4 1 2 3
Total number of bug found 0 0 0 0


Category Pilot Participant Participant 1 Participant 2 Participant 3
Navigation/Login/Overall look
  • Ugly layout for navigation bar
  • Poor navigation bar where links are hard to find
  • Suggest to have vertical links instead
  • Suggest for navigation bar links to be clicked instead of hover
  • Mouse-over very sensitive for the navigation bar
  • Navigation bar too long
  • Layout is not very nice
Remuneration
  • Not a need to have prompt message for update payment status
  • Suggest to have the Month the payslip is on to be at the top of the payslip
  • Suggest to include basic pay in the pay slip
  • Payslip description is a bit vague
  • Include basic pay in pay slip
  • Basic salary not shown in payslip
Claims
  • Upload button is redundant
  • Unclear table headings
  • Unable to view claim submitted
  • Incorrect naming of button of "Generate Pay Slip", it should be "Generate Claim Slip"
  • Add or remove row button inserted at the bottom of table
  • Button is wrongly named. Should be “Generate claim slip” instead
  • Adding new row to submit another claim is not clear
  • The value for the button is wrong. It should be Generate Claim slip
Leave
  • Cannot find arrow on selection on calendar
  • Inconsistent spelling
  • System should calculate the number of days taken and make it more obvious
  • Suggest "Confirm" button to calculate balance if leave is approved
  • Radio button look like bullet point
  • Radio button should be white when unclicked
  • Page is very clustered with a lot of information
  • Button size not standardize
  • Calendar is not working properly
  • Too many things cramped into one page
  • Naming not good
  • Button size not standardize

User Test 2

Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Pilot Participant Participant 1 Participant 2 Participant 3 Participant 4
ZORA UT2 PP.JPG
ZORA UT2 Ang.JPG
ZORA UT2 LC.JPG
ZORA UT2 Ro.JPG
ZORA UT2 FR.JPG
General Data
Age 35 45 36 31 35
Occupation Business Development Manager HR Personnel Contract Manager (HM) Structural Draftsman Quantity Surveyor
Test Duration 1 hour 30 mins 1 hour 40 mins 1 hour 10 mins 1 hour 20 mins 1 hour 25 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) Y N N N N
Claims (Quantitative Data Analysis)
Time taken to submit new claims 59 seconds 2 minutes 36 seconds 2 minutes 30 seconds 1 minute 45 seconds 1 minute 35 seconds
Time taken to validate submitted claims 1 minute 27 seconds 2 minutes 9 seconds 1 minute 40 seconds 2 minutes 20 seconds 2 minutes 10 seconds
Time taken to edit payment method 41 seconds 1 minutes 18 seconds 24 seconds 45 seconds 36 seconds
Time taken to approve/ reject submitted claims 56 seconds 3 minutes 29 seconds 2 minutes 6 seconds 1 minute 30 seconds 2 minutes 20 second
Time taken to generate claim slip 26 seconds N/A N/A N/A N/A
Time taken to view claim history 33 seconds N/A N/A N/A N/A
Time taken to view claim status 12 seconds 21 seconds 22 seconds 25 seconds 28 seconds
Total number of clicks to finish this module 22 25 28 30 34
Total number of question asked 5 9 3 6 10
Total number of bug found 4 0 0 1 0
Leave (Quantitative Data Analysis)
Time taken to apply annual leave 33 seconds 2 minutes 16 seconds 1 minute 1 second 1 minute 15 seconds 1 minute 40 seconds
Time taken to edit leave 33 seconds 1 minute 32 seconds 22 seconds 35 seconds 40 seconds
Time taken to apply other leave N/A 5 minutes 8 seconds 1 minute 45 seconds 1 minute 35 seconds 2 minutes
Time taken to approve/reject leave 1 minute 27 seconds 3 minutes 18 seconds 1 minute 23 seconds 1 minute 1 minute 50 seconds
Time taken to view status of leave & cancel leave 20 seconds 59 seconds 51 seconds 1 minute 15 seconds 3 minutes
Time taken to view leave history 20 seconds 33 seconds 11 seconds 20 seconds 15 seconds
Time taken to record medical leave 1 minute 30 seconds 3 minutes 30 seconds 2 minutes 6 seconds 2 minutes 20 seconds 2 minutes 10 seconds
Total number of clicks to finish this module 64 65 58 62 60
Total number of question asked 4 11 1 6 10
Total number of bug found 1 1 1 1 0
Remuneration (Quantitative Data Analysis)
Time taken to view past payment record for employees N/A 22 seconds 10 seconds 18 seconds N/A
Time taken to view past payment record for workers 4 seconds 14 seconds 5 seconds 10 seconds N/A
Time taken to view past payment record for drivers 1 seconds 7 seconds 24 seconds 9 seconds N/A
Time taken to view past payment record for workers N/A 22 seconds 51 seconds 35 seconds N/A
Time taken to generate all payment record for employees N/A 22 seconds 51 seconds 35 seconds N/A
Time taken to generate all payment record for workers N/A 26 seconds 7 seconds 15 seconds N/A
Time taken to generate all payment record for drivers N/A 15 seconds 12 seconds 10 seconds N/A
Time taken to view personal payment record N/A 36 seconds 27 seconds 24 seconds N/A
Time taken to input timesheet for drivers 2 minutes 14 seconds 1 minute 31 seconds 1 minute 9 seconds 2 minute N/A
Time taken to input timesheet for workers N/A 3 minute 5 seconds 2 minute 19 seconds 2 minute 30 seconds N/A
Total number of clicks to finish this module 60 65 60 66 N/A
Total number of question asked 1 1 0 1 N/A
Total number of bug found 1 5 5 1 N/A
Performance Management & Appraisal (Quantitative Data Analysis)
Time taken to create employees appraisal 45 seconds 2 minutes 36 seconds 1 minute 47 seconds 2 minutes 1 minute 30 seconds
Time taken to approve/revise salary increment requests 23 seconds 26 seconds 29 seconds 1 minute 45 seconds 1 minute 15 seconds
Time taken to distribute bonus 50 seconds 2 minutes 38 seconds 2 minutes 24 seconds 50 seconds 45 seconds
Time taken to view scheduled increment/bonus N/A 19 seconds 8 seconds 10 seconds 12 seconds
Time taken to view increment history N/A 24 seconds 12 seconds 10 seconds 12 seconds
Total number of clicks to finish this module 43 40 45 41 39
Total number of question asked 2 7 3 3 3
Total number of bug found 0 0 0 0 0
System Administration (Quantitative Data Analysis)
Time taken to create new profile 4 minutes 3 seconds 7 minutes 37 seconds 3 minutes 53 seconds 6 minutes 5 minutes 20 seconds
Time taken to update profile 2 minutes 12 seconds 2 minutes 26 seconds 1 minute 30 seconds 1 minute 34 seconds
Time taken to manage default number of leave 35 seconds 1 minute 40 seconds 1 minute 20 seconds 45 seconds 35 seconds
Total number of clicks to finish this module 37 40 38 36 35
Total number of question asked 1 0 0 0 0
Total number of bug found 1 1 0 0 0
Category Pilot Participant Participant 1 Participant 2 Participant 3 Participant 4
Navigation/Login/Overall look
  • Fonts need to be bigger
  • User-friendly with nothing too overwhelming
  • Description and titles are simple and not too wordy
  • Find it difficult to access the pages
  • Generally fluid with navigation
  • Make the date format in a format everyone is used to (I.e. dd/mm/yyyy)
  • Application is helpful and easy to use
Claim
  • N/A
  • Had difficulty finding the receipt for viewing
  • Did not know the need to hit "Update" after editing the amount
  • Hard to locate the claim receipt
  • N/A
  • N/A
Leave
  • N/A
  • Did not notice the dropdown in Record Leave
  • Did not know how to upload more than 1 file
  • N/A
  • N/A
Remuneration
  • PDF can have more space
  • N/A
  • N/A
  • Minus sign in the pay slip for those value that is going to be deducted
  • Add date for every signature
  • N/A
Performance & Appraisal
  • N/A
  • N/A
  • N/A
  • Approve does not automatically save the new value, and also allow to revise the value just in case the value is wrong
  • The title for view scheduled increments/bonus seemed confusing
  • N/A
System Administration
  • N/A
  • Colour of username is available should not be red in color
  • Ajax is not working in Firefox
  • Have a dropdown to select department
  • Have validation for phone numbers and postal code
  • N/A
  • N/A

Response Plan (Implement)

S/N Functionality Feedback Proposed Solution Proposed Decision
Claim
1 Submit Claim User tend to click "Add Claim" instead of "Submit Claim" Will shift the "Add Claim" button to somewhere else" Implement
2 validate Claim & Approve/Reject Claim User cannot find where to view the claim receipt Make the "view" link bigger and more distinct Implement
Leave
1 Apply Leave User is unsure of how to upload more than 1 files while applying for leave that require supporting documents Have a short instruction before the upload file field (i.e. Hold down CTRL button to select multiple options) Implement
2 Apply Leave User is unsure of what to put in the "Date To" field to apply leave on the same day When user selects the date in "Date From" field, the same date should be repopulated in the "Date To" field Implement
3 Apply Leave/Record Leave User tends to type the date instead of choosing the date using the datepicker leading to the wrong format being input Have a short instruction before the "Date To" and "Date From" field to indicate the date format and give an example of it Implement
4 User should be able to view leave of their colleagues in the same department so that they can plan how to cover up leave Have a separate "View Leave" page for users to view leave of people from the same department Make the "view" link bigger and more distinct Implement
Remuneration
1 View Personal Payslip tab User feel that it is hard to navigate to view personal payment record link when it is at the bottom Change the arrangement for the links under remuneration tab Implement
2 View payslip User finds that the less part of the payslip can be more obvious with a "-" sign Include the "-" sign in the PDF payslip Implement
3 View payslip Instead of month only, should include year as well in the payslip Include the year Implement
Performance & Appraisal
1 Edit Salary Increment User does not click the "Save" button when revising the salary increment amont. As a result, the amount approved is not the revised amount. It is not obvious that the "Save" button should be clicked before submitting the form In the event the increment value is not saved, the user should not be allowed to approve the increment request Implement
2 View Scheduled Salary Increment/Scheduled Bonus Currently the "Scheduled salary increment" and "Scheduled Bonuses" tables reside on the same page ("View Scheduled Increments/Bonus"). This can be confusing to the user as he/she is unable to distinguish between scheduled salary increments and scheduled bonuses Change the colours of the tables to differentiate between scheduled salary increments and scheduled bonuses Implement
3 Submit Appraisal Placement of "Submit Appraisal" and "Clear" buttons. Button placements are different from other modules. Not intuitive and users may get confused Standardize position of buttons across modules Implement
4 Submit Appraisal When an appaisal has been submitted successfully, it is not obvious to the user Display a successful appraisal submission message Implement
System Administration
1 Manage Leave Change default no. of leave --> User unsure of change is successful cos there is no popup or anything Display a successful update message Implement
2 Create/Edit Profile Remind the user the DOB format to type. User unsure of what to type. The date thing also must select super long ago. super troublesome for them Display a date format beside the field Implement
Navigation
1 Navigation In the navigation bar, users tend to click on menu items which display sub-menus, this results in the menu options being minimize. Selection of sub-menu items require users to hover over the main menu items and then clicking the sub-menus items Enable the navigation bar to support both hover and click functionality. In both instances, the sub-menu will be displayed and the menu will not be minimized Implement
Login
1 Error message Error message for login is black in colour Error message for login should be red in colour Implement
2 Active status Account with inactive status can login Account with inactive status should not be able login Implement
Changes made from User Test 2 Feedbacks
Claims
550
Leave
550
Remuneration
550
Performance Management & Appraisal
550
Quantitative Feedback Analysis
Ease of Use
350
Overall Layout
350
User Test 2 Analysis

View Full Analysis.

Response Plan (Not Implementing)

S/N Functionality Feedback Reason for Not Implementing Proposed Decision
Leave
1 Approve/Reject Leave People in the department should have the authority of approving/rejecting the leave the user sends The sponsor of the project mentioned that the line manager should be the only one approving. He should know if anyone in the department rejects to that particular leave on a particular day in case of any project that arises. Hence, according to the business logic, only the line manager should have the authority to approve/reject leave Not Implementing
2 Apply Leave User takes quite a while to find the leave balance on the "Apply Leave" page As the task instruction explicitly mention to the tester to check on the leave balance, tester might not know that it can be checked in the "Apply Leave" page. Moreover, it is only 1 out of 5 of the user that mentioned that. However, everyone eventually managed to find the leave balance when they are applying leave Not Implementing
Remuneration
1 Payslip Suggestion: For payslip, put signature and date The payslip is followed closely to the current payslip that the company is using Not Implementing
Performance & Appraisal
1 Edit Appraisal User should have the option to edit Appraisal once submitted Client explicitly mentioned that Appraisals should not be edited once submitted Not Implementing
System Administration
1 Create/Edit Profile Creating user profile --> User tend to click Tab. Take note to rearrange input data values. E.g. Put "Given Name" beside "Family Name" cos that is where the cursor go when user click Tab. Do take note of this Not really important Not Implementing
2 Create/Edit Profile To input allowance --> amount for which allowance type? If user select more than 1 allowance type, the $500 is for which one? It suppose to be like this, is a combine amount Not Implementing
Login/Logout
1 Logout User takes a few secs to find logout button Attributed to lack of familiarity with the system user interface. Not Implementing

User Test 3

In this user test, we did the testing on 9 participants, of which 5 participants are the same person from user test 2. This is so that we are able to do comparison between the qualitative and quantitative results gathered from user test 2 with the results that we gathered in this user test 3. Due to the space constraints, we will be showing the results gathered from the 4 new participants. If you want to view more results from user test 3, please click on this -> View Full Analysis.

Results obtained from the testing conducted The following table shows the summary results for quantitative data analysis in terms of the number of time taken for various functions, clicks, bugs and question asked:

Data Participant 1 Participant 2 Participant 3 Participant 4
Zora UT3 Participant 6.png
ZORA UT3 CHING.JPG
ZORA UT3 RAS.jpg
ZORA UT3 STU.JPG
General Data
Age 28 30 30 20
Occupation Design Engineer Quantity Surveyor Work & Safety Health Officer Student Intern
Test Duration 1 hour 15 mins 1 hour 1 hour 4 mins 1 hour 30 mins
Browser Google Chrome Google Chrome Google Chrome Google Chrome
Experienced in HRMS (Y/N) N N N N
Claims (Quantitative Data Analysis)
Time taken to submit new claims 1 minute 27 seconds 37 seconds 1 minute 1 minute 54 seconds
Time taken to validate submitted claims 1 minute 21 seconds 1 minutes 14 seconds 2 minute 26 seconds 1 minutes 8 seconds
Time taken to edit payment method 22 seconds 15 seconds 37 seconds 43 seconds
Time taken to approve/ reject submitted claims 25 seconds 29 seconds 58 seconds 42 seconds
Time taken to generate claim slip 2 minutes 32 seconds 59 seconds 2 minutes 56 seconds 40 seconds
Time taken to view claim history 1 minute 14 seconds 30 seconds 41 seconds 20 seconds
Time taken to view claim status 27 seconds 20 seconds 51 seconds 9 seconds
Time taken to generate claim report 30 seconds 25 seconds 40 seconds 27 seconds
Total number of question asked 0 4 7 0
Total number of bug found 0 0 0 0
Leave (Quantitative Data Analysis)
Time taken to apply & edit annual leave 2 minutes 37 seconds 1 minutes 19 seconds 2 minute 36 second 1 minute 50 seconds
Time taken to apply other leave 2 minutes 55 seconds 2 minutes 1 minute 44 seconds 1 minute
Time taken to approve leave 2 minutes 36 seconds 37 seconds 3 minutes 15 seconds 2 minutes 10 seconds
Time taken to view status of leave & cancel leave 1 minute 24 seconds 55 seconds 1 minute 20 seconds 2 minute 4 seconds
Time taken to view leave history 10 seconds 10 seconds 1 minute 25 seconds 46 seconds
Time taken to record medical leave 2 minutes 47 seconds 1 minute 44 seconds 1 minute 45 seconds 2 minutes 25 seconds
Time taken to record unpaid leave & approve recorded leave 2 minutes 51 seconds 1 minutes 50 seconds 3 minutes 5 seconds 2 minutes 10 seconds
Time taken to generate leave report 15 seconds 56 seconds 2 minutes 35 seconds 15 seconds
Total number of question asked 1 2 6 3
Total number of bug found 0 0 0 0
Remuneration (Quantitative Data Analysis)
Time taken to generate all payment record for employees 1 minute 12 seconds 18 seconds 45 seconds 52 seconds
Time taken to view past payment record for employees 7 seconds 20 seconds 49 seconds 13 seconds
Time taken to view personal payment record 33 seconds 16 seconds 55 seconds 20 seconds
Time taken to input timesheet for workers 3 minutes 19 seconds 1 minutes 18 seconds 2 minutes 45 seconds 3 minutes 23 seconds
Time taken to input OT timesheet for drivers 1 minute 18 seconds 23 seconds 1 minute 47 seconds 3 minute 3 seconds
Time taken to view worker costing 13 seconds 5 seconds 1 minute 19 seconds 15 seconds
Total number of question asked 2 0 4 3
Total number of bug found 0 0 0 0
Performance Management & Appraisal (Quantitative Data Analysis)
Time taken to appraise employees 3 minutes 21 seconds 58 seconds 2 minute 21 seconds 3 minutes 19 seconds
Time taken to distribute bonus 1 minute 11 seconds 30 seconds 1 minute 55 seconds 52 seconds
Total number of question asked 0 0 4 1
Total number of bug found 0 0 0 0
Training & Development (Quantitative Data Analysis)
Time taken to propose training 1 minutes 25 seconds 35 seconds 3 minutes 1 minutes 57 second
Time taken to approve/reject training by HR 3 minutes 43 seconds 1 minute 55 seconds 6 minutes 3 seconds 2 minutes 55 seconds
Time taken to approve/reject training by MD 49 seconds 30 seconds 1 minute 44 seconds 55 seconds
Time taken to view proposed training status 26 seconds 14 seconds 47 seconds 1 minute 7 seconds
Time taken to view nominated training 1 minute 3 seconds 38 seconds 1 minute 5 seconds 21 seconds
Time taken to manage external training 1 minute 2 seconds 32 seconds 1 minute 35 seconds 1 minute 4 seconds
Total number of question asked 0 0 5 0
Total number of bug found 0 0 0 0
System Administration (Quantitative Data Analysis)
Time taken to manage default number of leave 1 minutes 41 seconds 24 seconds 1 minutes 22 seconds 31 seconds
Time taken to create new profile 5 minutes 58 seconds 5 minutes 14 seconds 3 minute 55 seconds 4 minute 59 seconds
Time taken to update profile 46 seconds 51 seconds 1 minute 7 seconds 1 minute 12 seconds
Total number of question asked 0 0 5 1
Total number of bug found 0 0 0 0

Response Plan (Implement)

S/N Functionality Feedback Proposed Solution Proposed Decision
Claim
1 Submit Claim Tedious to record receipt numbers of the claim receipts by HR Have an addition field "Receipt No." for employee to key in their own receipt number and HR will just validate instead of having to input it herself Implement
2 Validate Claim Did not know need to update the edited amount before continuing to validate the claim To include a "message" whenever the edit button is click to remind the user that update button must be hit before the changes can be made Implement
3 Approve/Reject Claim Admin accept/reject the status not obvious Make it bigger and more striking colour Implement
4 Edit Payment Method Have a back button to generate claimslip Implement "Go to Generate Claimslip" button below the edit payment method page Implement
5 Generate Claimslip No need claimslip for employee Take out the employee's copy of claimslip in the PDF generated Implement
6 Generate Claimslip Make the slip bigger Make the slip bigger Implement
7 Generate Claimslip More space for signature Spread the claim slip to have more space and include a date space to input date Implement
Leave
1 Record Leave Not obvious that employee's name is able to be selected by keying the alphabet Enable filtering for user's to type employee's name Implement
2 Emailing for all Leave Leave approval emails did not CC HR personnel CC HR personnel for all leave emails Implement
3 Approve/Reject Leave Never state day duration in email (e.g. half day) State day duration of leave taken in email Implement
Remuneration
1 Input worker timesheet Full name of workers should be shown for worker input timesheet page Insert full name of workers Implement
Performance & Appraisal
1 View full appraisal Text on "View Appraisal" button is not specific Change Button Text to "View Full Appraisal" Implement
2 View full appraisal Scores displayed are not meaningful to the user Display of Remuneration Scores- show total (e.g. 100/100) Implement
Training & Development
1 Approve/Reject Proposed Training UI Changes for select the training to be approve/reject. Accept / Reject button unclear, user wasn't sure she needs to click the button to proceed Change the name of the button Implement
2 Approve/Reject Proposed Training UI Changes for the page that they input the training details. Back button not standardize. No need write "default". Venue nv put "*". Instead of add session, it should be add another training session. Different form of writing time (e.g. 9am vs 0900. Course duration give dropdown for hrs/mins Standardize back button. Remove the default wording. Venue is a required. Change value for the button for adding training session. Put the indication that is a 24 hour (so accept numbers only). A dropdown box that have the hr/mins/day/month/year for course duration Implement
3 Propose Training Need to check if the employee have or didn't have any people under him/her. Nomination of employee Do a check if there is no employee under him, it will prompt him a message Implement
4 Manage External Training Training Status from Completed change to Course Completed Change from completed to Course Completed Implement
System Administration
1 Manage Employee Adding the department dropdown list Add a table in db + and add a dropdown list Implement
Navigation
1 General Font size could be larger Increase Font Size Implement
Changes made from User Test 3 Feedbacks
Claims Before
550


After
550
Leave Before
550


After
550
Training & Development Before
550


After
550
Quantitative Feedback - User Test 2 versus User Test 3
Ease of Use
Zora Ease of use analysis.png


Overall Layout
Zora overall layout analysis.png


Analysis for the same 5 participants from user test 2

This is the analysis done from user test 2:

Zora UT3 Before Analysis.png

Reasons for the time taken:

  • Validate Submitted Claim - Most users are not able to locate the "view" claim receipt link
  • Edit Payment Method - Confirmation message was not display thus user did not know the payment method had been updated
  • Apply Other Leave - Most users was unsure how to upload 2 files as no instruction was stated hence a huge amount of time was spent figuring it out


This is the analysis done for user test 3:

Zora UT3 After Analysis.png

Reasons for changes in timing:

  • Changes that we made and improved on the application was useful
  • Live Deployment + user interacting with the system before enables them to familiar with the system

Response Plan (Not Implementing)

S/N Functionality Feedback Reason for Not Implementing Proposed Decision
Leave
1 Record medical leave Did not check for duplicate medical series no There is a possibility that 2 person can have the same medical series no. Hence, this will not be checked Not Implementing
Remuneration
1 Worker payslip Timesheet to include dinner breakdown (3h x 2) instead of giving the total ending sum Dinner is allocated $3 for each dinner, if necessary. The payslip in the system is mirroring the current payslip they are using Not Implementing
2 Input worker timesheet User tend to miss out the days but will notice when see days The timesheet in the system is mirroring the current worker timesheet they used. Others may not be familiar but the HR personnel who is in charged of inputting the timesheet will be familiar if it is the same format as the one she is using Not Implementing
Performance & Appraisal
1 Create Employee Appraisal Radio button too small. Cannot click properly Radio buttons are controlled by system and browser. Customization of radio buttons would not enable usage on mobile devices Not Implementing
Training & Development
1 Manage External Training Suggest user can type "UOB" for cheque no. (so allow text and no.) Cheque number got nothing to do with the Bank name Not Implementing
System Administration
1 Manage Employee Date Cannot go back to earlier than 2014. Wage Type, Hourly, Edit button on the top, cause too much to scroll down. The requirement that we had agreed on earlier was to either put in daily wage or monthly wage Not Implementing