HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T2 5 B 1 G Usability Documentation"

From IS480
Jump to navigation Jump to search
 
(27 intermediate revisions by 2 users not shown)
Line 42: Line 42:
  
 
| style="padding:2px; font-size:130%;  background-color:#ffffff; text-align:center; color:#ffffff" width="10%" |   
 
| style="padding:2px; font-size:130%;  background-color:#ffffff; text-align:center; color:#ffffff" width="10%" |   
[[IS480_Team_wiki:_2012T2_5_B_1_G Retailer Admin System UTs | <font color="#000000">Retailer Admin System UTs</font>]]
+
[[IS480_Team_wiki:_2012T2_5_B_1_G Webapp User Tests | <font color="#000000">Webapp User Tests</font>]]
  
 
| style="padding:2px; font-size:130%;  background-color:#ffffff; text-align:center; color:#ffffff" width="10%" |   
 
| style="padding:2px; font-size:130%;  background-color:#ffffff; text-align:center; color:#ffffff" width="10%" |   
Line 50: Line 50:
  
 
<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; font-size:190%; margin-top: 25px; line-height: 1.2em; color:white">Summary of User Tests Documentation</div>
 
<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; font-size:190%; margin-top: 25px; line-height: 1.2em; color:white">Summary of User Tests Documentation</div>
 
  
 
{| class="wikitable" style="text-align: center"
 
{| class="wikitable" style="text-align: center"
Line 56: Line 55:
 
|-
 
|-
 
! scope="col" width="150" style="background:#CCCCCC| User Test  
 
! scope="col" width="150" style="background:#CCCCCC| User Test  
! scope="col" width="150" style="background:#CCCCCC| Date Conducted
+
! scope="col" width="150" style="background:#CCCCCC| Details
 +
! scope="col" width="150" style="background:#CCCCCC| Methodology
 
! scope="col" width="150" style="background:#CCCCCC| Documents for User Test
 
! scope="col" width="150" style="background:#CCCCCC| Documents for User Test
 
! scope="col" width="150" style="background:#CCCCCC| User Test Results
 
! scope="col" width="150" style="background:#CCCCCC| User Test Results
Line 62: Line 62:
 
|-
 
|-
 
|[[#User_Test_1| 1]]
 
|[[#User_Test_1| 1]]
|align="center"| 2 Nov 2012 (Fri)
+
|align="left"| Date: 2 Nov 2012 (Fri) <br> Participants: 7 shoppers <br>Location: SMU, SIS UPL Lab <br>
 +
|align="center"| Paper Prototype
 
|align="center"| [[Media:5B1G_UT1_Execution_Plan.pdf|UT 1 Execution Plan]]
 
|align="center"| [[Media:5B1G_UT1_Execution_Plan.pdf|UT 1 Execution Plan]]
 
|align="center"| [[Media:5B1G_UT1_Results.pdf|UT 1 Results]]<br>
 
|align="center"| [[Media:5B1G_UT1_Results.pdf|UT 1 Results]]<br>
Line 68: Line 69:
 
|-
 
|-
 
|[[#User_Test_2| 2]]
 
|[[#User_Test_2| 2]]
|align="center"| 14 Jan 2013 (Mon)
+
|align="left"| Date: 14 Jan 2013 (Mon) <br> Participants: 10 shoppers <br> Location: SMU, SIS UPL Lab
 +
|align="center"| Paper Prototype
 
|align="center"| [[Media:5B1G_UT2_Execution Plan.pdf|UT 2 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G_UT2_Execution Plan.pdf|UT 2 Execution Plan]] <br>
 
[[Media:5B1G_UT2_Test_sheet.xlsx|UT 2 Test Sheet]] <br>
 
[[Media:5B1G_UT2_Test_sheet.xlsx|UT 2 Test Sheet]] <br>
Line 75: Line 77:
 
|-
 
|-
 
|[[#User_Test_3| 3]]
 
|[[#User_Test_3| 3]]
|align="center"| 15 Feb 2013 (Mon)
+
|align="left"| Date: 15 Feb 2013 (Mon) <br> Participants: 14 shoppers <br> Location: SMU SIS SR 3-4 <br>
 +
|align="center"| (App testing) Goals were given for users to complete
 
|align="center"| [[Media:UT3_Execution_Plan.pdf|UT 3 Execution Plan]] <br>
 
|align="center"| [[Media:UT3_Execution_Plan.pdf|UT 3 Execution Plan]] <br>
 
[[Media:UT3_Instruction_sheet.pdf‎|UT 3 Instruction Sheet]] <br>
 
[[Media:UT3_Instruction_sheet.pdf‎|UT 3 Instruction Sheet]] <br>
Link for Survey [http://smusg.asia.qualtrics.com/SE/?SID=SV_5pPvWIbXkCVcFPD here]
+
Link for Survey [http://smusg.asia.qualtrics.com/SE/?SID=SV_7Wc6VjyNdaONEm9 here]
 
|align="center"| [[Media:5B1G_UT3_Raw_Qualitative_Results.xlsx| UT 3 Raw Qualitative Results]] <br>
 
|align="center"| [[Media:5B1G_UT3_Raw_Qualitative_Results.xlsx| UT 3 Raw Qualitative Results]] <br>
 
[[Media:UT3_Results.pdf| UT 3 Compiled Results]]
 
[[Media:UT3_Results.pdf| UT 3 Compiled Results]]
 
|-
 
|-
 
|[[#User_Test_4| 4]]
 
|[[#User_Test_4| 4]]
|align="center"| 11 Mar 2013 (Mon)
+
|align="left"| Date: 11 Mar 2013 (Mon) <br> Participants: 15 shoppers <br> Location:SR 2-3 (starting & ending point),GSR 3-5 (Pedro Shop), GSR 2-4 (The Basement)
 +
|align="center"| (App testing) Goals were given for users to complete in a simulated real environment
 
|align="center"| [[Media:5B1G_UT4_Execution_Plan.pdf|UT 4 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G_UT4_Execution_Plan.pdf|UT 4 Execution Plan]] <br>
 
[[Media: UT4_Android_Instruction_sheet.pdf| UT4 Android Instructions Sheet]]<br>
 
[[Media: UT4_Android_Instruction_sheet.pdf| UT4 Android Instructions Sheet]]<br>
Line 89: Line 93:
 
[[Media:5B1G_UT4_Results.pdf| UT 4 Compiled Results]]
 
[[Media:5B1G_UT4_Results.pdf| UT 4 Compiled Results]]
 
|-
 
|-
|[[#User_Test_5| 5: SMU, The Basement Cafe]]
+
|[[#User_Test_5| 5]]
|align="center"| 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues)
+
|align="left"| Date: 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues) <br> Participants: 24 shoppers <br> Location: The Basement Café
 +
|align="center"| (App testing) Goals were given for users to complete in a real setting
 
|align="center"| [[Media:5B1G_UT5_with_The_Basement_Cafe.pdf|UT 5 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G_UT5_with_The_Basement_Cafe.pdf|UT 5 Execution Plan]] <br>
 
[[Media: UT5_Instruction_sheet_(Facilitator).pdf| UT5 Android Instructions Sheet (Facilitator)]]<br>
 
[[Media: UT5_Instruction_sheet_(Facilitator).pdf| UT5 Android Instructions Sheet (Facilitator)]]<br>
Line 98: Line 103:
 
[[Media:UT5_Results.pdf|UT 5 Compiled Results]]
 
[[Media:UT5_Results.pdf|UT 5 Compiled Results]]
 
|-
 
|-
|[[#User_Test_6_at_The_Cathay| 6: The Cathay]]
+
|[[#User_Test_6_.28The_Cathay.29| 6]]
|align="center"| 11-12 April 2013 (Thursday and Friday)
+
|align="left"| Date: 11 Apr 2013 (Thurs) & 12 Apr 2013 (Fri) <br>Participants: 22 shoppers <br> Location: The Cathay
 +
|align="center"| Field Testing (under actual conditions)
 
|align="center"| [[Media:UT6_at_The_Cathay.pdf|UT6 at The Cathay Execution Plan]] <br>
 
|align="center"| [[Media:UT6_at_The_Cathay.pdf|UT6 at The Cathay Execution Plan]] <br>
 
Link for Survey [http://smusg.asia.qualtrics.com/SE/?SID=SV_82OaL3JlHpOnttj here] <br>
 
Link for Survey [http://smusg.asia.qualtrics.com/SE/?SID=SV_82OaL3JlHpOnttj here] <br>
 
[[Media:UT6_Instruction_sheet.pdf| IPS Instructions Sheet]]
 
[[Media:UT6_Instruction_sheet.pdf| IPS Instructions Sheet]]
 
|align="center"| [[Media:UT6_Results.pdf| UT6 Results]]
 
|align="center"| [[Media:UT6_Results.pdf| UT6 Results]]
|-
 
|[[#Web_User_Test_1|Webapp UT1]]
 
|align="center"| 22 Jan 2013 (Tues)
 
|align="center"| [[Media:Web_UT1_Execution_Plan.pdf|Web UT1 Execution Plan]]
 
|align="center"| [[Media: UTWeb_Results.pdf| Web UT1 Results]]
 
|-
 
|[[#Web_User_Test_2|Webapp UT2]]
 
|align="center"| 25 Mar 2013 (Mon)
 
|align="center"| [[Media:Web_UT2_Execution_Plan.pdf|Web UT2 Execution Plan]] <br>
 
[[Media:Web_UT2_Instruction_sheet.pdf| Web UT2 Instructions Sheet]]<br>
 
|align="center"| [[Media:WebUT2_Results.pdf| Web UT2 Compiled Results]]
 
|-
 
|[[#Web_User_Test_3|Webapp UT3]]
 
|align="center"| 10 April 2013 (Wednesday)
 
|align="center"| [[Media:5B1G_Web_UT3_Execution_Plan.pdf|Web UT3 Execution Plan]] <br>
 
[[Media:Web_UT3_Instruction_Sheet.pdf| Web UT3 Instructions Sheet]]<br>
 
|align="center"| [[Media:WebUT3_Results.pdf| Web UT3 Compiled Results]]
 
|-
 
|[[#Web_User_Test_4|Webapp UT4]]
 
|align="center"| 17 April 2013 (Wednesday)
 
|align="center"| [[Media:5B1G Web_UT4_Execution_Plan.pdf|Web UT4 Execution Plan]] <br>
 
[[Media:Web_UT4_Instructions_Sheet.pdf| Web UT4 Instructions Sheet]]<br>
 
|align="center"| [[Media:WebUT4_Results.pdf| Web UT4 Compiled Results]]
 
|-
 
|[[#Admin_Web_UT|Admin Web UT]]
 
|align="center"| 16 April 2013 (Tuesday)
 
|align="center"| [[Media:Admin_Web_UT_Execution_Plan.pdf|Admin Web UT Execution Plan]] <br>
 
[[Media:Admin_Web_UT_Instructions_Sheet.pdf| Admin Web UT Instructions Sheet]]
 
|align="center"| [[Media:Admin_Web_UT_Results.pdf| Admin Web System Compiled Results]]
 
 
|-
 
|-
 
|}
 
|}
 
 
  
 
'''Test Plan''' <br>
 
'''Test Plan''' <br>
 
Download the test plan [[Media:5B1G Test Plan.pdf| here]]
 
Download the test plan [[Media:5B1G Test Plan.pdf| here]]
 +
 +
'''Collated Quantitative Results:''' <br>
 +
*[[Media:UT Collated Results.xlsx|Collated UT Results]]
 +
*[[Media:Web_UT_Collated_Results.xlsx|Collated Web UT Results]]
  
 
'''Compatibility Information (NFC enabled device):''' <br>
 
'''Compatibility Information (NFC enabled device):''' <br>
Line 655: Line 634:
 
*To stress-test the application in a condition where many users will be using the application
 
*To stress-test the application in a condition where many users will be using the application
  
*To gather feedback from our users with regards to the functionalities of the application and
+
*To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
 
 
the process flow of the concept
 
  
 
====Details: ====
 
====Details: ====
Line 813: Line 790:
  
  
=<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Webapp User Test 3 (Cathay retailers) </div>=
 
 
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">How We Conducted the UT</div>===
 
 
====Objectives====
 
*To obtain feedback from our retailers with regards to the usability of the retailer admin system
 
*To create an application that is user-friendly for retailers
 
  
====Details: ====
 
*Participants: 6 retailers
 
*Venue: The Cathay
 
*Date: 10th Apr 2013 (Wedneday)
 
*Duration: 6 hours (each individual: 20-30mins)
 
 
====Method====
 
The retailer admin system was tested using a laptop . Each participant was given the [[media: Web_UT3_Instruction_sheet.pdf| Web UT3 Instruction Sheet]] and asked to circle how easy it was to complete each test case.
 
 
====Data collection====
 
Collecting of qualitative metrics
 
*Facilitators will observe how the participants use the retail admin system during the testing procedure.
 
*Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
 
 
Collecting of quantitative metrics
 
* Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
 
 
====Features Tested for Retail Admin System====
 
{| class="wikitable" style="text-align: center; height:25px"
 
|+
 
|-
 
! scope="col"  width="50" style="background:#CCCCCC| S/N
 
! scope="col" width="250" style="background:#CCCCCC| Feature
 
|-
 
!scope="row"  style=" text-align: center;"| 1
 
| Create a retail account
 
|-
 
!scope="row"  style=" text-align: center;"| 2
 
| Login
 
|-
 
!scope="row"  style=" text-align: center;"| 3
 
| Upload new E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 4
 
| Edit Pending E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 5
 
| Copy E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 6
 
| Delete E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 7
 
| View published E-coupon statistics
 
|-
 
!scope="row"  style=" text-align: center;"| 8
 
| Delete E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 9
 
| Logout
 
|-
 
|}
 
 
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Results</div>===
 
 
====Download the results here: ====
 
* [[Media:5B1G_WebUT3_Results.pdf| Web UT3 Compiled Results]]
 
 
====Compiled Problems & Proposed Solutions====
 
 
[[Image:5B1G UT 3 problem.png|580px]]
 
  
 
=<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">User Test 6 (The Cathay)</div>=
 
=<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">User Test 6 (The Cathay)</div>=
Line 938: Line 847:
 
[[Media:UT6_Results.pdf| UT6 Results]]
 
[[Media:UT6_Results.pdf| UT6 Results]]
  
=<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Admin Web UT</div>=
+
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Gallery</div>===
  
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">How We Conducted the UT</div>===
+
{| class="wikitable" style="text-align: center; height:130px"
 +
! scope="col" width="100" style="background:none;border-top: 1px solid #ffffff; border-right: 1px solid #ffffff; border-left: 1px solid #ffffff"|
 +
| style="border-bottom:1px solid #ffffff; border-top:1px ; background:none;border: 1px solid #ffffff" width="5%" | &nbsp;
 +
! scope="col" width="100" style="background:none;border-top: 1px solid #ffffff; border-right: 1px solid #ffffff; border-left: 1px solid #ffffff"|
 +
| style="border-bottom:1px solid #ffffff; border-top:1px solid #ffffff; background:none;border: 1px solid #ffffff" width="5%" | &nbsp;
 +
! scope="col" width="100" style="background:none;border-top: 1px solid #ffffff; border-right: 1px solid #ffffff; border-left: 1px solid #ffffff"|
 +
|-
  
====Objectives====
+
!scope="row"  style=" text-align: center;"|
*To obtain feedback from our with regards to the usability of the admin web system
+
[[Image:5B1G_UT6_1.jpg|280px]]
*To create an application that is user-friendly for administrators
+
| style="border-bottom:1px solid #ffffff; background: #ffffff; color: #FFFFFF; font-weight: bold; padding: 10px 15px 0 15px; font-size: 26px;" valign="top"|
 +
|  color: #000000; padding: 0px 4px 0 0px; font-size: 13px;" valign="top" align="center"  |
 +
[[image: 5B1G_UT6_5.jpg|280px]]
 +
| style="border-bottom:1px solid #ffffff; background: #ffffff; color: #FFFFFF; font-weight: bold; padding: 10px 15px 0 15px; font-size: 26px;" valign="top"|
 +
|  color: #000000; padding: 0px 4px 0 0px; font-size: 13px;" valign="top" align="center"  |
 +
[[image:5B1G_UT6_3.jpg|280px]]
 +
|-
  
====Details: ====
+
!scope="row"  style=" text-align: center;"|
*Participants: 6 retailers
+
[[Image: 5B1G_UT6_4.jpg|280px]]
*Venue: The Cathay
+
| style="border-bottom:1px solid #ffffff; background: #ffffff; color: #FFFFFF; font-weight: bold; padding: 10px 15px 0 15px; font-size: 26px;" valign="top"|
*Date: 16th Apr 2013 (Tuesday)
+
|  color: #000000; padding: 0px 4px 0 0px; font-size: 13px;" valign="top" align="center"  |
*Duration: 6 hours (each individual: 20-30mins)
+
[[image: 5B1G_UT6_2.jpg|280px]]
 +
| style="border-bottom:1px solid #ffffff; background: #ffffff; color: #FFFFFF; font-weight: bold; padding: 10px 15px 0 15px; font-size: 26px;" valign="top"|
 +
|  color: #000000; padding: 0px 4px 0 0px; font-size: 13px;" valign="top" align="center"  |
 +
[[image:5B1G_UT6_6.jpg|280px]]
 +
|-
  
====Method====
+
!scope="row" style=" text-align: center;"|
The admin web system was tested using a laptop . Each participant was given the [[media: 5B1G admin web instruction sheet.pdf| Admin Web Instruction Sheet]] and asked to circle how easy it was to complete each test case.
+
[[Image:5B1G_UT6_7.jpg |280px]]
 
+
| style="border-bottom:1px solid #ffffff; background: #ffffff; color: #FFFFFF; font-weight: bold; padding: 10px 15px 0 15px; font-size: 26px;" valign="top"|
====Data collection====
+
| color: #000000; padding: 0px 4px 0 0px; font-size: 13px;" valign="top" align="center" |
Collecting of qualitative metrics
+
[[image: 5B1G_UT6_8.jpg|280px]]
*Facilitators will observe how the participants use the admin web system during the testing procedure.
+
| style="border-bottom:1px solid #ffffff; background: #ffffff; color: #FFFFFF; font-weight: bold; padding: 10px 15px 0 15px; font-size: 26px;" valign="top"|
*Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
+
color: #000000; padding: 0px 4px 0 0px; font-size: 13px;" valign="top" align="center" |
 
+
[[image:5B1G_UT6_9.jpg|280px]]
Collecting of quantitative metrics
 
* Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
 
 
 
====Features Tested for Admin Web System====
 
{| class="wikitable" style="text-align: center; height:25px"
 
|+
 
|-
 
! scope="col"  width="50" style="background:#CCCCCC| S/N
 
! scope="col" width="250" style="background:#CCCCCC| Feature
 
|-
 
!scope="row" style=" text-align: center;"| 1
 
| Login
 
|-
 
!scope="row"  style=" text-align: center;"| 2
 
| View existing retailers
 
|-
 
!scope="row" style=" text-align: center;"| 3
 
| Delete existing retailer
 
|-
 
!scope="row" style=" text-align: center;"| 4
 
| Approve pending retailer
 
 
|-
 
|-
!scope="row"  style=" text-align: center;"| 5
 
|Toggle  between self developed IPS and Indoo.rs
 
|-
 
!scope="row"  style=" text-align: center;"| 6
 
| Logout
 
 
|}
 
 
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Results</div>===
 
 
====Download the results here: ====
 
* [[Media:5B1G Admin_Web_UT_results.pdf| Admin Web System Compiled Results]]
 
 
=<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#009999', endColorstr='#009999'); background: -webkit-gradient(linear, left top, left bottom, from(#009999), to(#009999)); background: -moz-linear-gradient(top, #009999, #009999); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Webapp User Test 4(Cathay retailers)</div>=
 
 
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">How We Conducted the UT</div>===
 
 
====Objectives====
 
*To obtain feedback from our retailers with regards to the usability of the retailer admin system with the updated changes from the last user test
 
 
====Details: ====
 
*Participants: 6 retailers
 
*Venue: The Cathay
 
*Date: 17th Apr 2013 (Wedneday)
 
*Duration: 6 hours (each individual: 20-30mins)
 
 
====Method====
 
The retailer admin system was tested using a laptop . Each participant was given the [[media: 5B1G_Web_UT4_Instruction_sheet.pdf| Web UT4 Instruction Sheet]] and asked to circle how easy it was to complete each test case.
 
 
====Data collection====
 
Collecting of qualitative metrics
 
*Facilitators will observe how the participants use the retail admin system during the testing procedure.
 
*Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
 
  
Collecting of quantitative metrics
 
* Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
 
 
====Features Tested for Retail Admin System====
 
{| class="wikitable" style="text-align: center; height:25px"
 
|+
 
|-
 
! scope="col"  width="50" style="background:#CCCCCC| S/N
 
! scope="col" width="250" style="background:#CCCCCC| Feature
 
|-
 
!scope="row"  style=" text-align: center;"| 1
 
| Create a retail account
 
|-
 
!scope="row"  style=" text-align: center;"| 2
 
| Login
 
|-
 
!scope="row"  style=" text-align: center;"| 3
 
| Upload new E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 4
 
| Crop new E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 5
 
| Edit Pending E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 6
 
| Copy E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 7
 
| Delete E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 8
 
| View Published E-coupon statistics
 
|-
 
!scope="row"  style=" text-align: center;"| 9
 
| View or export BI report
 
|-
 
!scope="row"  style=" text-align: center;"| 10
 
| Delete E-coupon
 
|-
 
!scope="row"  style=" text-align: center;"| 11
 
| Logout
 
|-
 
 
|}
 
|}
 
===<div style="width: 100%; filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00BBBB', endColorstr='#00BBBB'); background: -webkit-gradient(linear, left top, left bottom, from(#00BBBB), to(#00BBBB)); background: -moz-linear-gradient(top, #00BBBB, #00BBBB); padding:3px; margin-top: 15px; line-height: 1.2em; color:white">Results</div>===
 
 
====Download the results here: ====
 
* [[Media:WebUT4_Results.pdf| Web UT4 Compiled Results]]
 

Latest revision as of 16:54, 28 July 2013

5B1G Logo.png


Home

Our Team/Sponsors

Project Overview

Project Management

Project Documentation

Usability Documentation

Learning Outcomes


Count Me In User Tests

Webapp User Tests

Prototypes & Mockups

Summary of User Tests Documentation
User Test Details Methodology Documents for User Test User Test Results
1 Date: 2 Nov 2012 (Fri)
Participants: 7 shoppers
Location: SMU, SIS UPL Lab
Paper Prototype UT 1 Execution Plan UT 1 Results

UT 1 Video

2 Date: 14 Jan 2013 (Mon)
Participants: 10 shoppers
Location: SMU, SIS UPL Lab
Paper Prototype UT 2 Execution Plan

UT 2 Test Sheet
UT 2 User Appreciation

UT 2 Results
3 Date: 15 Feb 2013 (Mon)
Participants: 14 shoppers
Location: SMU SIS SR 3-4
(App testing) Goals were given for users to complete UT 3 Execution Plan

UT 3 Instruction Sheet
Link for Survey here

UT 3 Raw Qualitative Results

UT 3 Compiled Results

4 Date: 11 Mar 2013 (Mon)
Participants: 15 shoppers
Location:SR 2-3 (starting & ending point),GSR 3-5 (Pedro Shop), GSR 2-4 (The Basement)
(App testing) Goals were given for users to complete in a simulated real environment UT 4 Execution Plan

UT4 Android Instructions Sheet

UT 4 Raw Qualitative Results

UT 4 Compiled Results

5 Date: 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues)
Participants: 24 shoppers
Location: The Basement Café
(App testing) Goals were given for users to complete in a real setting UT 5 Execution Plan

UT5 Android Instructions Sheet (Facilitator)
UT5 Instructions Sheet (User)
Link for Survey here

UT 5 Raw Qualitative Results

UT 5 Compiled Results

6 Date: 11 Apr 2013 (Thurs) & 12 Apr 2013 (Fri)
Participants: 22 shoppers
Location: The Cathay
Field Testing (under actual conditions) UT6 at The Cathay Execution Plan

Link for Survey here
IPS Instructions Sheet

UT6 Results

Test Plan
Download the test plan here

Collated Quantitative Results:

Compatibility Information (NFC enabled device):
Samsung S3, Samsung Galaxy Note 2
Samsung S2*, HTC One*
*These devices are unable to carry out the NFC redemption

Our application is able to run on Ice Cream Sandwich (API level 15) onwards

Incompatible with the following device:
HTC Desire | Froyo (API level 8)
MotoBlur | Froyo (API level 8)

Contents

User Test 1

How We Conducted the UT

Objectives

To understand whether the user finds the usability of the application user-friendly The table below shows a list of features that will tested for our paper prototype user test to test the usability of the new layout

Details:

  • Participants: 7 shoppers/ undergraduates
  • Venue: SMU, SIS UPL Lab
  • Date: 2nd Nov 2012 (Friday)
  • Duration: 1 hour

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.

Features Tested

S/N Feature
1 Sign in as an existing user
2 Login with Facebook
3 Bring me there
4 Share me (share on facebook)
5 Bookmark me
6 Scan a product and add to cart
7 View my shopping cart/checkout
8 View past transactions
9 Browse coupons
10 View recommendations

Results

Observations

95%

Feedback

95%

Gallery

Videos:
UT 1 Video
5B1G UT 1.PNG

Changes Made

Change 1

Change 2

Change 3

Change 4

Change 5

User Test 2

How We Conducted the UT

Objectives

The main objectives of this user test are:

  • To obtain feedback from our users with regards to the layout in our application, and
  • To implement a layout that is considered to be of acceptable standard by users

Details:

  • Participants: 10 shoppers/undergraduates
  • Venue: SMU, SIS UPL Lab
  • Date: 14th Jan 2013
  • Duration: 2 hours

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
  • We will compile the problems and classify them using a set of heuristics, thereafter we will assign a severity rating to each compiled problem

Features Tested

S/N Feature
1 Login with facebook
2 Push notification, view recommended coupons
3 View home page/featured coupons
4 Share me (share on facebook)
5 Bring me there (navigation)
6 Add to Coupon Cart
7 Use coupon (NFC tag)
8 Browse coupon (by category & location)
9 Map
10 Coupon Cart
11 Logout of application

Results

Detailed Results

Download here: UT 2 Results

Compiled Problems & Proposed Solutions

5B1G UT2 Problem1.png

5B1G UT2 Problem2.png

5B1G UT2 Problem3.png

5B1G UT2 Problem4.png

5B1G UT2 Problem5.png


Gallery

   

5B1G UT2 4.jpg

5B1G UT2 5.jpg

5B1G UT2 6.jpg

5B1G UT2 1.png

5B1G UT2 2.png

5B1G UT2 3.png

Changes Made (during the UT)

Initial paper prototype Changes made during UT2
5B1G arrow paper prototype.jpg

Most participants failed to realize that the arrow on the top left corner was clickable

5B1G Arrow changes.JPG

Changed the icon to a more obvious and clickable icon that does not that does
that does not look like part of the app logo


User Test 3

How We Conducted the UT

Objectives

  • To test the functionalities of the application from a user's perspective
  • To obtain feedback from our users with regards to the usability of the application

Details:

  • Participants: 14 shoppers/ undergraduates
  • Venue: SMU, SIS SR 3-4
  • Date: 15th Feb 2013 (Friday)
  • Duration: 3 hours

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the instruction sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe the how quickly the participant is able to complete the test case and whether he or she meets with any confusion (such as clicking on the wrong item) during each test case. These observations will be recorded down

Collecting of quantitative metrics

  • Upon completion of each test case, the participant will be asked to rate how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested

S/N Feature
1 Login with Facebook
2 View Home Paage/ Latest e-coupons
3 Share Me (share on Facebook)
4 Coupon Cart
5 Use Me (NFC tag)
6 Search Coupon (search by category and location
7 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT3 Problem1.png

5B1G UT3 Problem2.png

5B1G UT3 Problem3.png

5B1G UT3 Problem4.png

5B1G UT3 Problem5.png

Gallery

   

5B1G UT3 1.JPG

5B1G UT3 2.JPG

5B1G UT3 3.JPG

5B1G UT3 4.JPG

5B1G UT3 5.jpg

5B1G UT3 6.JPG

5B1G UT3 7.jpg

5B1G UT3 8.JPG

5B1G UT3 9.JPG

5B1G UT3 13.jpg

5B1G UT3 10.JPG

5B1G UT3 11.JPG

5B1G UT3 14.JPG

5B1G UT3 16.JPG

5B1G UT3 18.JPG

User Test 4

How We Conducted the UT

Objectives

  • To test the functionalities of the both the android application and the web application in a simulated real life environment
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept

Details:

  • Participants: 15 shoppers/ undergraduates
  • Venue:
    • Testing venues for Count Me In android app:
      • SR 2-3 (starting & ending point)
      • GSR 3-5 (Pedro Shop)
      • GSR 2-4 (The Basement)
  • Date: 11th Mar 2013 (Monday)
  • Duration: 4 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
  • Users will be asked to give remarks for each test case.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for “Count Me In”

S/N Feature
1 Login with Facebook
2 View Latest E-coupons
3 Share Me
4 Add to Favourites
5 Push notification to view recommended E-coupons
6 Indoor Navigation
7 Receive location-specific push notification & view recommended e-coupons
8 Use Me
9 Bring Me
10 Delete E-coupon
11 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT4 Problem1.png

5B1G UT4 Problem2.png

Gallery

   

5B1G UT4 1.jpg

5B1G UT4 2.JPG

5B1G UT4 3.JPG

5B1G UT4 10.JPG

5B1G UT4 8.JPG

5B1G UT4 9.JPG

User Test 5

How We Conducted the UT

Objectives

  • To test the functionalities of the android application in a real life environment
  • To stress-test the application in a condition where many users will be using the application
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept

Details:

  • Participants: 24 shoppers/ undergraduates
  • Venue: The Basement Café
  • Date: 18th Mar 2013 (Monday), 19th Mar 2013 (Tuesday)
  • Duration: 4.5 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
  • Users will be asked to give remarks for each test case.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for “Count Me In”

S/N Feature
1 Login with Facebook
2 View Latest E-coupons
3 Indoor Navigation
4 Bring Me
5 Receive location-specific push notification & view Recommended
6 Share Me
7 Favourite Me
8 View Categories
9 Delete E-coupons
10 Use Me
11 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT5 Problem1.png

Gallery

   

5B1G 4.jpg

5B1G 5.jpg

5B1G 6.jpg

5B1G 7.JPG

5B1G 8.JPG

5B1G 9.JPG

5B1G 11.JPG

5B1G 13.JPG

5B1G 14.JPG

5B1G 15.JPG

5B1G 16.JPG

5B1G 17.JPG

5B1G 18.JPG

5B1G 19.JPG

5B1G 20.jpg

5B1G 21.JPG

5B1G 23.JPG

5B1G 24.JPG

5B1G 1.JPG

5B1G 2.jpg

5B1G 3.jpg



User Test 6 (The Cathay)

How We Conducted the UT

Objectives

We will be testing Count Me In android application at The Cathay and hope to achieve the following objectives:

  • To test and gather feedback on Count Me In in a deployed environment
  • To test the Cosine Similarity Index algorithm for the recommendation
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
  • To test the IPS with these test cases: UT6 Instructions Sheet

As the user test will be conducted in a deployed real life environment, no instructions sheet will be provided for the user for features that have already been tested. The user test is largely focused on the pushing of recommended E-coupons based on the user’s region.

Details:

  • Participants: 22 shoppers (18 undergraduates 4 public)
  • Venue: The Cathay
  • Participating Retail Outlets:
    • Popeyes
    • Createaf Waffles
    • Gong Cha
    • Marcella
    • St. Games
    • Jshoes
  • Date: 11th Apr 2013 (Thursday) & 12 Apr 2013 (Friday)
  • Duration: 4 hours each day (each individual: 20-30mins)

Method

As this is a field test taken under the conditions of actual operation, no instructions sheet was given to the users. Instead, the users were asked to download the application, redeem and E-coupon and then proceed to complete a survey giving their feedback on whether the app was easy to use and whether they would continue to use it. The only instructions that was given was pertaining to the indoor positioning system which had not been tested. For this field test, we invited students who belong to the targeted group of 'shoppers' and approached people from the public to download and use the application.

Data collection

Collecting of qualitative metrics

  • Conducted a survey of users who
  • Users will be asked to give remarks for each test case.

Features Tested (we tested those features that have not been tested)

S/N Feature
1 Find Retailer
1 Indoor Map

Results

Download the results here:

UT6 Results

Gallery

   

5B1G UT6 1.jpg

5B1G UT6 5.jpg

5B1G UT6 3.jpg

5B1G UT6 4.jpg

5B1G UT6 2.jpg

5B1G UT6 6.jpg

5B1G UT6 7.jpg

5B1G UT6 8.jpg

5B1G UT6 9.jpg