IS480 Team wiki: 2012T2 5 B 1 G Usability Documentation
Test Plan
Download the test plan here
Compatibility Information (NFC enabled device):
Samsung S3, Samsung Galaxy Note 2
Samsung S2*, HTC One*
*These devices are unable to carry out the NFC redemption
Our application is able to run on Ice Cream Sandwich (API level 15) onwards
Incompatible with the following device:
HTC Desire | Froyo (API level 8)
MotoBlur | Froyo (API level 8)
Contents
User Test 1
How We Conducted the UT
Objectives
To understand whether the user finds the usability of the application user-friendly The table below shows a list of features that will tested for our paper prototype user test to test the usability of the new layout
Details:
- Participants: 7 shoppers/ undergraduates
- Venue: SMU, SIS UPL Lab
- Date: 2nd Nov 2012 (Friday)
- Duration: 1 hour
Method
We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.
Data collection
Collecting of qualitative metrics
- We will observe how the participants use the application during the testing procedure.
- Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
Features Tested
S/N | Feature |
---|---|
1 | Sign in as an existing user |
2 | Login with Facebook |
3 | Bring me there |
4 | Share me (share on facebook) |
5 | Bookmark me |
6 | Scan a product and add to cart |
7 | View my shopping cart/checkout |
8 | View past transactions |
9 | Browse coupons |
10 | View recommendations |
Results
Observations
Feedback
Gallery
Videos:
UT 1 Video
Changes Made
Change 1
Change 2
Change 3
Change 4
Change 5
User Test 2
How We Conducted the UT
Objectives
The main objectives of this user test are:
- To obtain feedback from our users with regards to the layout in our application, and
- To implement a layout that is considered to be of acceptable standard by users
Details:
- Participants: 10 shoppers/undergraduates
- Venue: SMU, SIS UPL Lab
- Date: 14th Jan 2013
- Duration: 2 hours
Method
We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.
Data collection
Collecting of qualitative metrics
- We will observe how the participants use the application during the testing procedure.
- Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
- We will compile the problems and classify them using a set of heuristics, thereafter we will assign a severity rating to each compiled problem
Features Tested
S/N | Feature |
---|---|
1 | Login with facebook |
2 | Push notification, view recommended coupons |
3 | View home page/featured coupons |
4 | Share me (share on facebook) |
5 | Bring me there (navigation) |
6 | Add to Coupon Cart |
7 | Use coupon (NFC tag) |
8 | Browse coupon (by category & location) |
9 | Map |
10 | Coupon Cart |
11 | Logout of application |
Results
Detailed Results
Download here: UT 2 Results
Compiled Problems & Proposed Solutions
Gallery
Changes Made (during the UT)
Webapp User Test
How We Conducted the UT
Objectives
The main objectives of this user test are:
- To obtain feedback from our users with regards to the layout and business logic in our application
Details:
- Participants: 2 retailers
- Venue: SMU, The Basement Café
- Date: 22nd Jan 2013 (Tuesday)
- Duration: 2 hours
- Environment: “Sit at a table” setting
Method
We conducted the user test using a low-fidelity paper prototype using the mock up designs of the web application and the participants were given the freedom to explore the application by themselves.
Data collection
Collecting of qualitative metrics
- We will observe how the participants use the application during the testing procedure.
- Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and business rules behind the application.
Features Tested
S/N | Feature |
---|---|
1 | Login with an account |
2 | Upload e-coupons |
3 | Edit pending e-coupons |
4 | View published e-coupons |
5 | View expired e-coupons |
6 | View Business Intelligence for published and expired e-coupons |
7 | Logout of application |
Gallery
Results
Results
Download link available here: Web UT Results
Observations
S/N | Observation |
---|---|
1 | The user did not have any major problems regarding the usability the application |
2 | They were more concerned about the business logic, they asked about whether they need to upload or select an individual product, because sometimes they offer discounts like “10% off all mains” rather than discount for an individual product |
3 | Overall, they were quite fascinated by the analytics that the application can produce |
Feedback
S/N | Feedback |
---|---|
1 | “Tagging an e-coupon to a product may not be always applicable, because sometimes we just want to entice people to come and patronize the restaurants, thus we may have a promotion for bulk items” |
2 | “Maybe you could limit the number of coupons we can give out because sometimes when we conduct an offer, we do not have sufficient stock” |
Changes Made
Features Tested
S/N | Feedback | Change Made |
---|---|---|
1 | “Tagging an e-coupon to a product may not be always applicable, because
sometimes we just want to entice people to come and patronize the restaurants, thus we may have a promotion for bulk items” |
The user does not need to tag a product to the coupon. We reduced it to a 1-step upload process instead of a 2-step upload process. |
2 | ““Maybe you could limit the number of coupons we can give out because
sometimes when we conduct an offer, we do not have sufficient stock” |
We included a new field for the retailer to enter the limit on the number of coupons (optional) |
User Test 3
How We Conducted the UT
Objectives
- To test the functionalities of the application from a user's perspective
- To obtain feedback from our users with regards to the usability of the application
Details:
- Participants: 14 shoppers/ undergraduates
- Venue: SMU, SIS SR 3-4
- Date: 15th Feb 2013 (Friday)
- Duration: 3 hours
Method
The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the instruction sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe the how quickly the participant is able to complete the test case and whether he or she meets with any confusion (such as clicking on the wrong item) during each test case. These observations will be recorded down
Collecting of quantitative metrics
- Upon completion of each test case, the participant will be asked to rate how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested
S/N | Feature |
---|---|
1 | Login with Facebook |
2 | View Home Paage/ Latest e-coupons |
3 | Share Me (share on Facebook) |
4 | Coupon Cart |
5 | Use Me (NFC tag) |
6 | Search Coupon (search by category and location |
7 | Logout of application |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
User Test 4
How We Conducted the UT
Objectives
- To test the functionalities of the both the android application and the web application in a simulated real life environment
- To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
Details:
- Participants: 15 shoppers/ undergraduates
- Venue:
- Testing venues for Count Me In android app:
- SR 2-3 (starting & ending point)
- GSR 3-5 (Pedro Shop)
- GSR 2-4 (The Basement)
- Testing venues for Count Me In android app:
- Date: 11th Mar 2013 (Monday)
- Duration: 4 hours (each individual: 20-30mins)
Method
The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
- Users will be asked to give remarks for each test case.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for “Count Me In”
S/N | Feature |
---|---|
1 | Login with Facebook |
2 | View Latest E-coupons |
3 | Share Me |
4 | Add to Favourites |
5 | Push notification to view recommended E-coupons |
6 | Indoor Navigation |
7 | Receive location-specific push notification & view recommended e-coupons |
8 | Use Me |
9 | Bring Me |
10 | Delete E-coupon |
11 | Logout of application |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
User Test 5
How We Conducted the UT
Objectives
- To test the functionalities of the android application in a real life environment
- To stress-test the application in a condition where many users will be using the application
- To gather feedback from our users with regards to the functionalities of the application and
the process flow of the concept
Details:
- Participants: 24 shoppers/ undergraduates
- Venue: The Basement Café
- Date: 18th Mar 2013 (Monday), 19th Mar 2013 (Tuesday)
- Duration: 4.5 hours (each individual: 20-30mins)
Method
The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
- Users will be asked to give remarks for each test case.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for “Count Me In”
S/N | Feature |
---|---|
1 | Login with Facebook |
2 | View Latest E-coupons |
3 | Indoor Navigation |
4 | Bring Me |
5 | Receive location-specific push notification & view Recommended |
6 | Share Me |
7 | Favourite Me |
8 | View Categories |
9 | Delete E-coupons |
10 | Use Me |
11 | Logout of application |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
Webapp User Test 2
How We Conducted the UT
Objectives
- To obtain feedback from our users with regards to the usability of the retail admin system
- To create an application that is user-friendly for retailers
Details:
- Participants: 10 users
- Venue: SIS SR 2-4
- Date: 25th Mar 2013 (Monday)
- Duration: 4 hours (each individual: 20-30mins)
Method
The actual application was tested this time using Google Chrome. Each participant was given the Web UT2 Instruction Sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use the retail admin system during the testing procedure.
- Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for Retail Admin System
S/N | Feature |
---|---|
1 | Create a retail account |
2 | Login |
3 | Upload new E-coupon |
4 | Edit Pending E-coupon |
5 | Copy E-coupon |
6 | Delete E-coupon |
7 | View Published E-coupon statistics |
8 | Logout |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
Webapp User Test 3 (with Cathay retailers)
How We Conducted the UT
Objectives
- To obtain feedback from our retailers with regards to the usability of the retailer admin system
- To create an application that is user-friendly for retailers
Details:
- Participants: 6 retailers
- Venue: The Cathay
- Date: 10th Apr 2013 (Wedneday)
- Duration: 6 hours (each individual: 20-30mins)
Method
The retailer admin system was tested using a laptop . Each participant was given the Web UT3 Instruction Sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use the retail admin system during the testing procedure.
- Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for Retail Admin System
S/N | Feature |
---|---|
1 | Create a retail account |
2 | Login |
3 | Upload new E-coupon |
4 | Edit Pending E-coupon |
5 | Copy E-coupon |
6 | Delete E-coupon |
7 | View Published E-coupon statistics |
8 | Delete E-coupon |
9 | Logout |
Results
Download the results here:
Compiled Problems & Proposed Solutions
User Test 6 (The Cathay)
How We Conducted the UT
Objectives
We will be testing Count Me In android application at The Cathay and hope to achieve the following objectives:
- To test and gather feedback on Count Me In in a deployed environment
- To test the Cosine Similarity Index algorithm for the recommendation
- To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
- To test the IPS with these test cases: UT6 Instructions Sheet
As the user test will be conducted in a deployed real life environment, no instructions sheet will be provided for the user for features that have already been tested. The user test is largely focused on the pushing of recommended E-coupons based on the user’s region.
Details:
- Participants: 22 shoppers (18 undergraduates 4 public)
- Venue: The Cathay
- Participating Retail Outlets:
- Popeyes
- Createaf Waffles
- Gong Cha
- Marcella
- St. Games
- Jshoes
- Date: 11th Apr 2013 (Thursday) & 12 Apr 2013 (Friday)
- Duration: 4 hours each day (each individual: 20-30mins)
Method
As this is a field test taken under the conditions of actual operation, no instructions sheet was given to the users. Instead, the users were asked to download the application, redeem and E-coupon and then proceed to complete a survey giving their feedback on whether the app was easy to use and whether they would continue to use it. The only instructions that was given was pertaining to the indoor positioning system which had not been tested. For this field test, we invited students who belong to the targeted group of 'shoppers' and approached people from the public to download and use the application.
Data collection
Collecting of qualitative metrics
- Conducted a survey of users who
- Users will be asked to give remarks for each test case.
Features Tested (we tested those features that have not been tested)
S/N | Feature |
---|---|
1 | Find Retailer |
1 | Indoor Map |
Results
Download the results here:
Admin Web UT
How We Conducted the UT
Objectives
- To obtain feedback from our with regards to the usability of the admin web portal
- To create an application that is user-friendly for administrators
Details:
- Participants: 6 retailers
- Venue: The Cathay
- Date: 16th Apr 2013 (Tuesday)
- Duration: 6 hours (each individual: 20-30mins)
Method
The admin web system was tested using a laptop . Each participant was given the Admin Web Instruction Sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use the retail admin system during the testing procedure.
- Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for Retail Admin System
S/N | Feature |
---|---|
1 | Login |
2 | View existing retailers |
3 | Delete existing retailer |
4 | Approving pending retailer |
5 | Toggle off and on IPS |
6 | Logout |
Results
Download the results here:
Web User Test 4(with Cathay retailers)
How We Conducted the UT
Objectives
- To obtain feedback from our retailers with regards to the usability of the retailer admin system with the updated changes from the last user test
Details:
- Participants: 6 retailers
- Venue: The Cathay
- Date: 17th Apr 2013 (Wedneday)
- Duration: 6 hours (each individual: 20-30mins)
Method
The retailer admin system was tested using a laptop . Each participant was given the Web UT4 Instruction Sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use the retail admin system during the testing procedure.
- Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for Retail Admin System
S/N | Feature |
---|---|
1 | Create a retail account |
2 | Login |
3 | Upload new E-coupon |
4 | Crop new E-coupon |
5 | Edit Pending E-coupon |
6 | Copy E-coupon |
7 | Delete E-coupon |
8 | View Published E-coupon statistics |
9 | View or export BI report |
10 | Delete E-coupon |
11 | Logout |