IS480 Team wiki: 2012T2 5 B 1 G Usability Documentation
User Test | Details | Methodology | Documents for User Test | User Test Results |
---|---|---|---|---|
1 | Date: 2 Nov 2012 (Fri) Participants: 7 shoppers Location: SMU, SIS UPL Lab |
Paper Prototype | UT 1 Execution Plan | UT 1 Results |
2 | Date: 14 Jan 2013 (Mon) Participants: 10 shoppers Location: SMU, SIS UPL Lab |
Paper Prototype | UT 2 Execution Plan |
UT 2 Results |
3 | Date: 15 Feb 2013 (Mon) Participants: 14 shoppers Location: SMU SIS SR 3-4 |
(App testing) Goals were given for users to complete | UT 3 Execution Plan UT 3 Instruction Sheet |
UT 3 Raw Qualitative Results |
4 | Date: 11 Mar 2013 (Mon) Participants: 15 shoppers Location:SR 2-3 (starting & ending point),GSR 3-5 (Pedro Shop), GSR 2-4 (The Basement) |
(App testing) Goals were given for users to complete in a simulated real environment | UT 4 Execution Plan |
UT 4 Raw Qualitative Results |
5 | Date: 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues) Participants: 24 shoppers Location: The Basement Café |
(App testing) Goals were given for users to complete in a real setting | UT 5 Execution Plan UT5 Android Instructions Sheet (Facilitator) |
UT 5 Raw Qualitative Results |
6 | Date: 11 Apr 2013 (Thurs) & 12 Apr 2013 (Fri) Participants: 22 shoppers Location: The Cathay |
Field Testing (under actual conditions) | UT6 at The Cathay Execution Plan Link for Survey here |
UT6 Results |
Test Plan
Download the test plan here
"'Collated Quantitative Results: ""
Compatibility Information (NFC enabled device):
Samsung S3, Samsung Galaxy Note 2
Samsung S2*, HTC One*
*These devices are unable to carry out the NFC redemption
Our application is able to run on Ice Cream Sandwich (API level 15) onwards
Incompatible with the following device:
HTC Desire | Froyo (API level 8)
MotoBlur | Froyo (API level 8)
Contents
User Test 1
How We Conducted the UT
Objectives
To understand whether the user finds the usability of the application user-friendly The table below shows a list of features that will tested for our paper prototype user test to test the usability of the new layout
Details:
- Participants: 7 shoppers/ undergraduates
- Venue: SMU, SIS UPL Lab
- Date: 2nd Nov 2012 (Friday)
- Duration: 1 hour
Method
We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.
Data collection
Collecting of qualitative metrics
- We will observe how the participants use the application during the testing procedure.
- Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
Features Tested
S/N | Feature |
---|---|
1 | Sign in as an existing user |
2 | Login with Facebook |
3 | Bring me there |
4 | Share me (share on facebook) |
5 | Bookmark me |
6 | Scan a product and add to cart |
7 | View my shopping cart/checkout |
8 | View past transactions |
9 | Browse coupons |
10 | View recommendations |
Results
Observations
Feedback
Gallery
Videos:
UT 1 Video
Changes Made
Change 1
Change 2
Change 3
Change 4
Change 5
User Test 2
How We Conducted the UT
Objectives
The main objectives of this user test are:
- To obtain feedback from our users with regards to the layout in our application, and
- To implement a layout that is considered to be of acceptable standard by users
Details:
- Participants: 10 shoppers/undergraduates
- Venue: SMU, SIS UPL Lab
- Date: 14th Jan 2013
- Duration: 2 hours
Method
We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.
Data collection
Collecting of qualitative metrics
- We will observe how the participants use the application during the testing procedure.
- Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
- We will compile the problems and classify them using a set of heuristics, thereafter we will assign a severity rating to each compiled problem
Features Tested
S/N | Feature |
---|---|
1 | Login with facebook |
2 | Push notification, view recommended coupons |
3 | View home page/featured coupons |
4 | Share me (share on facebook) |
5 | Bring me there (navigation) |
6 | Add to Coupon Cart |
7 | Use coupon (NFC tag) |
8 | Browse coupon (by category & location) |
9 | Map |
10 | Coupon Cart |
11 | Logout of application |
Results
Detailed Results
Download here: UT 2 Results
Compiled Problems & Proposed Solutions
Gallery
Changes Made (during the UT)
User Test 3
How We Conducted the UT
Objectives
- To test the functionalities of the application from a user's perspective
- To obtain feedback from our users with regards to the usability of the application
Details:
- Participants: 14 shoppers/ undergraduates
- Venue: SMU, SIS SR 3-4
- Date: 15th Feb 2013 (Friday)
- Duration: 3 hours
Method
The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the instruction sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe the how quickly the participant is able to complete the test case and whether he or she meets with any confusion (such as clicking on the wrong item) during each test case. These observations will be recorded down
Collecting of quantitative metrics
- Upon completion of each test case, the participant will be asked to rate how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested
S/N | Feature |
---|---|
1 | Login with Facebook |
2 | View Home Paage/ Latest e-coupons |
3 | Share Me (share on Facebook) |
4 | Coupon Cart |
5 | Use Me (NFC tag) |
6 | Search Coupon (search by category and location |
7 | Logout of application |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
User Test 4
How We Conducted the UT
Objectives
- To test the functionalities of the both the android application and the web application in a simulated real life environment
- To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
Details:
- Participants: 15 shoppers/ undergraduates
- Venue:
- Testing venues for Count Me In android app:
- SR 2-3 (starting & ending point)
- GSR 3-5 (Pedro Shop)
- GSR 2-4 (The Basement)
- Testing venues for Count Me In android app:
- Date: 11th Mar 2013 (Monday)
- Duration: 4 hours (each individual: 20-30mins)
Method
The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
- Users will be asked to give remarks for each test case.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for “Count Me In”
S/N | Feature |
---|---|
1 | Login with Facebook |
2 | View Latest E-coupons |
3 | Share Me |
4 | Add to Favourites |
5 | Push notification to view recommended E-coupons |
6 | Indoor Navigation |
7 | Receive location-specific push notification & view recommended e-coupons |
8 | Use Me |
9 | Bring Me |
10 | Delete E-coupon |
11 | Logout of application |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
User Test 5
How We Conducted the UT
Objectives
- To test the functionalities of the android application in a real life environment
- To stress-test the application in a condition where many users will be using the application
- To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
Details:
- Participants: 24 shoppers/ undergraduates
- Venue: The Basement Café
- Date: 18th Mar 2013 (Monday), 19th Mar 2013 (Tuesday)
- Duration: 4.5 hours (each individual: 20-30mins)
Method
The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.
Data collection
Collecting of qualitative metrics
- Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
- Users will be asked to give remarks for each test case.
Collecting of quantitative metrics
- Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.
Features Tested for “Count Me In”
S/N | Feature |
---|---|
1 | Login with Facebook |
2 | View Latest E-coupons |
3 | Indoor Navigation |
4 | Bring Me |
5 | Receive location-specific push notification & view Recommended |
6 | Share Me |
7 | Favourite Me |
8 | View Categories |
9 | Delete E-coupons |
10 | Use Me |
11 | Logout of application |
Results
Download the results here:
Compiled Problems & Proposed Solutions
Gallery
User Test 6 (The Cathay)
How We Conducted the UT
Objectives
We will be testing Count Me In android application at The Cathay and hope to achieve the following objectives:
- To test and gather feedback on Count Me In in a deployed environment
- To test the Cosine Similarity Index algorithm for the recommendation
- To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
- To test the IPS with these test cases: UT6 Instructions Sheet
As the user test will be conducted in a deployed real life environment, no instructions sheet will be provided for the user for features that have already been tested. The user test is largely focused on the pushing of recommended E-coupons based on the user’s region.
Details:
- Participants: 22 shoppers (18 undergraduates 4 public)
- Venue: The Cathay
- Participating Retail Outlets:
- Popeyes
- Createaf Waffles
- Gong Cha
- Marcella
- St. Games
- Jshoes
- Date: 11th Apr 2013 (Thursday) & 12 Apr 2013 (Friday)
- Duration: 4 hours each day (each individual: 20-30mins)
Method
As this is a field test taken under the conditions of actual operation, no instructions sheet was given to the users. Instead, the users were asked to download the application, redeem and E-coupon and then proceed to complete a survey giving their feedback on whether the app was easy to use and whether they would continue to use it. The only instructions that was given was pertaining to the indoor positioning system which had not been tested. For this field test, we invited students who belong to the targeted group of 'shoppers' and approached people from the public to download and use the application.
Data collection
Collecting of qualitative metrics
- Conducted a survey of users who
- Users will be asked to give remarks for each test case.
Features Tested (we tested those features that have not been tested)
S/N | Feature |
---|---|
1 | Find Retailer |
1 | Indoor Map |