HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T2 5 B 1 G Usability Documentation"

From IS480
Jump to navigation Jump to search
Line 128: Line 128:
 
|align="center"| 17 April 2013 (Wednesday)
 
|align="center"| 17 April 2013 (Wednesday)
 
|align="center"| [[Media:5B1G Web_UT4_Execution_Plan.pdf|Web UT4 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G Web_UT4_Execution_Plan.pdf|Web UT4 Execution Plan]] <br>
[[Media:Web_UT4_Instructions_Sheet.pdf|| Web UT4 Instructions Sheet]]<br>
+
[[Media:Web_UT4_Instructions_Sheet.pdf| Web UT4 Instructions Sheet]]<br>
 
|align="center"| [[Media:WebUT4_Results.pdf| Web UT4 Compiled Results]]
 
|align="center"| [[Media:WebUT4_Results.pdf| Web UT4 Compiled Results]]
 
|-
 
|-

Revision as of 16:24, 19 April 2013

5B1G Logo.png


Home

Our Team/Sponsors

Project Overview

Project Management

Project Documentation

Usability Documentation

Learning Outcomes


User Tests

Prototypes & Mockups

Summary of User Tests Documentation


User Test Date Conducted Documents for User Test User Test Results
1 2 Nov 2012 (Fri) UT 1 Execution Plan UT 1 Results

UT 1 Video

2 14 Jan 2013 (Mon) UT 2 Execution Plan

UT 2 Test Sheet
UT 2 User Appreciation

UT 2 Results
Web UT1 22 Jan 2013 (Tues) Web UT1 Execution Plan Web UT1 Results
3 15 Feb 2013 (Mon) UT 3 Execution Plan

UT 3 Instruction Sheet
Link for Survey here

UT 3 Raw Qualitative Results

UT 3 Compiled Results

4 11 Mar 2013 (Mon) UT 4 Execution Plan

UT4 Android Instructions Sheet

UT 4 Raw Qualitative Results

UT 4 Compiled Results

5: SMU, The Basement Cafe 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues) UT 5 Execution Plan

UT5 Android Instructions Sheet (Facilitator)
UT5 Instructions Sheet (User)
Link for Survey here

UT 5 Raw Qualitative Results

UT 5 Compiled Results

Web UT2 25 Mar 2013 (Mon) Web UT2 Execution Plan

Web UT2 Instructions Sheet

Web UT2 Compiled Results
Web UT3 10 April 2013 (Wednesday) Web UT3 Execution Plan

Web UT3 Instructions Sheet

Web UT3 Compiled Results
6: The Cathay 11-12 April 2013 (Thursday and Friday) UT6 at The Cathay Execution Plan

Link for Survey here
IPS Instructions Sheet

UT6 Results
Admin Web UT 16 April 2013 (Tuesday) Admin Web UT Execution Plan

Admin Web UT Instructions Sheet

Admin Web System Compiled Results
Web UT4 17 April 2013 (Wednesday) Web UT4 Execution Plan

Web UT4 Instructions Sheet

Web UT4 Compiled Results


Test Plan
Download the test plan here

Compatibility Information (NFC enabled device):
Samsung S3, Samsung Galaxy Note 2
Samsung S2*, HTC One*
*These devices are unable to carry out the NFC redemption

Our application is able to run on Ice Cream Sandwich (API level 15) onwards

Incompatible with the following device:
HTC Desire | Froyo (API level 8)
MotoBlur | Froyo (API level 8)

Contents

User Test 1

How We Conducted the UT

Objectives

To understand whether the user finds the usability of the application user-friendly The table below shows a list of features that will tested for our paper prototype user test to test the usability of the new layout

Details:

  • Participants: 7 shoppers/ undergraduates
  • Venue: SMU, SIS UPL Lab
  • Date: 2nd Nov 2012 (Friday)
  • Duration: 1 hour

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.

Features Tested

S/N Feature
1 Sign in as an existing user
2 Login with Facebook
3 Bring me there
4 Share me (share on facebook)
5 Bookmark me
6 Scan a product and add to cart
7 View my shopping cart/checkout
8 View past transactions
9 Browse coupons
10 View recommendations

Results

Observations

95%

Feedback

95%

Gallery

Videos:
UT 1 Video
5B1G UT 1.PNG

Changes Made

Change 1

Change 2

Change 3

Change 4

Change 5

User Test 2

How We Conducted the UT

Objectives

The main objectives of this user test are:

  • To obtain feedback from our users with regards to the layout in our application, and
  • To implement a layout that is considered to be of acceptable standard by users

Details:

  • Participants: 10 shoppers/undergraduates
  • Venue: SMU, SIS UPL Lab
  • Date: 14th Jan 2013
  • Duration: 2 hours

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
  • We will compile the problems and classify them using a set of heuristics, thereafter we will assign a severity rating to each compiled problem

Features Tested

S/N Feature
1 Login with facebook
2 Push notification, view recommended coupons
3 View home page/featured coupons
4 Share me (share on facebook)
5 Bring me there (navigation)
6 Add to Coupon Cart
7 Use coupon (NFC tag)
8 Browse coupon (by category & location)
9 Map
10 Coupon Cart
11 Logout of application

Results

Detailed Results

Download here: UT 2 Results

Compiled Problems & Proposed Solutions

5B1G UT2 Problem1.png

5B1G UT2 Problem2.png

5B1G UT2 Problem3.png

5B1G UT2 Problem4.png

5B1G UT2 Problem5.png


Gallery

   

5B1G UT2 4.jpg

5B1G UT2 5.jpg

5B1G UT2 6.jpg

5B1G UT2 1.png

5B1G UT2 2.png

5B1G UT2 3.png

Changes Made (during the UT)

Initial paper prototype Changes made during UT2
5B1G arrow paper prototype.jpg

Most participants failed to realize that the arrow on the top left corner was clickable

5B1G Arrow changes.JPG

Changed the icon to a more obvious and clickable icon that does not that does
that does not look like part of the app logo

Webapp User Test

How We Conducted the UT

Objectives

The main objectives of this user test are:

  • To obtain feedback from our users with regards to the layout and business logic in our application

Details:

  • Participants: 2 retailers
  • Venue: SMU, The Basement Café
  • Date: 22nd Jan 2013 (Tuesday)
  • Duration: 2 hours
  • Environment: “Sit at a table” setting

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the web application and the participants were given the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and business rules behind the application.

Features Tested

S/N Feature
1 Login with an account
2 Upload e-coupons
3 Edit pending e-coupons
4 View published e-coupons
5 View expired e-coupons
6 View Business Intelligence for published and expired e-coupons
7 Logout of application

Gallery

   

5B1G IMG 5073.JPG

5B1G IMG 5074.JPG

5B1G IMG 5075.JPG

IMG 5076.JPG

5B1G IMG 5077.JPG

5B1G IMG 5078.JPG

Results

Results

Download link available here: Web UT Results

Observations

S/N Observation
1 The user did not have any major problems regarding the usability the application
2 They were more concerned about the business logic, they asked about whether they need to upload or select an individual product, because sometimes they offer discounts like “10% off all mains” rather than discount for an individual product
3 Overall, they were quite fascinated by the analytics that the application can produce

Feedback

S/N Feedback
1 “Tagging an e-coupon to a product may not be always applicable, because sometimes we just want to entice people to come and patronize the restaurants, thus we may have a promotion for bulk items”
2 “Maybe you could limit the number of coupons we can give out because sometimes when we conduct an offer, we do not have sufficient stock”

Changes Made

Features Tested

S/N Feedback Change Made
1 “Tagging an e-coupon to a product may not be always applicable, because

sometimes we just want to entice people to come and patronize the restaurants, thus we may have a promotion for bulk items”

The user does not need to tag a product to the coupon. We reduced it to a 1-step upload process instead of a 2-step upload process.
2 ““Maybe you could limit the number of coupons we can give out because

sometimes when we conduct an offer, we do not have sufficient stock”

We included a new field for the retailer to enter the limit on the number of coupons (optional)

User Test 3

How We Conducted the UT

Objectives

  • To test the functionalities of the application from a user's perspective
  • To obtain feedback from our users with regards to the usability of the application

Details:

  • Participants: 14 shoppers/ undergraduates
  • Venue: SMU, SIS SR 3-4
  • Date: 15th Feb 2013 (Friday)
  • Duration: 3 hours

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the instruction sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe the how quickly the participant is able to complete the test case and whether he or she meets with any confusion (such as clicking on the wrong item) during each test case. These observations will be recorded down

Collecting of quantitative metrics

  • Upon completion of each test case, the participant will be asked to rate how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested

S/N Feature
1 Login with Facebook
2 View Home Paage/ Latest e-coupons
3 Share Me (share on Facebook)
4 Coupon Cart
5 Use Me (NFC tag)
6 Search Coupon (search by category and location
7 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT3 Problem1.png

5B1G UT3 Problem2.png

5B1G UT3 Problem3.png

5B1G UT3 Problem4.png

5B1G UT3 Problem5.png

Gallery

   

5B1G UT3 1.JPG

5B1G UT3 2.JPG

5B1G UT3 3.JPG

5B1G UT3 4.JPG

5B1G UT3 5.jpg

5B1G UT3 6.JPG

5B1G UT3 7.jpg

5B1G UT3 8.JPG

5B1G UT3 9.JPG

5B1G UT3 13.jpg

5B1G UT3 10.JPG

5B1G UT3 11.JPG

5B1G UT3 14.JPG

5B1G UT3 16.JPG

5B1G UT3 18.JPG

User Test 4

How We Conducted the UT

Objectives

  • To test the functionalities of the both the android application and the web application in a simulated real life environment
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept

Details:

  • Participants: 15 shoppers/ undergraduates
  • Venue:
    • Testing venues for Count Me In android app:
      • SR 2-3 (starting & ending point)
      • GSR 3-5 (Pedro Shop)
      • GSR 2-4 (The Basement)
  • Date: 11th Mar 2013 (Monday)
  • Duration: 4 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
  • Users will be asked to give remarks for each test case.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for “Count Me In”

S/N Feature
1 Login with Facebook
2 View Latest E-coupons
3 Share Me
4 Add to Favourites
5 Push notification to view recommended E-coupons
6 Indoor Navigation
7 Receive location-specific push notification & view recommended e-coupons
8 Use Me
9 Bring Me
10 Delete E-coupon
11 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT4 Problem1.png

5B1G UT4 Problem2.png

Gallery

   

5B1G UT4 1.jpg

5B1G UT4 2.JPG

5B1G UT4 3.JPG

5B1G UT4 10.JPG

5B1G UT4 8.JPG

5B1G UT4 9.JPG

User Test 5

How We Conducted the UT

Objectives

  • To test the functionalities of the android application in a real life environment
  • To stress-test the application in a condition where many users will be using the application
  • To gather feedback from our users with regards to the functionalities of the application and

the process flow of the concept

Details:

  • Participants: 24 shoppers/ undergraduates
  • Venue: The Basement Café
  • Date: 18th Mar 2013 (Monday), 19th Mar 2013 (Tuesday)
  • Duration: 4.5 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
  • Users will be asked to give remarks for each test case.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for “Count Me In”

S/N Feature
1 Login with Facebook
2 View Latest E-coupons
3 Indoor Navigation
4 Bring Me
5 Receive location-specific push notification & view Recommended
6 Share Me
7 Favourite Me
8 View Categories
9 Delete E-coupons
10 Use Me
11 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT5 Problem1.png

Gallery

   

5B1G 4.jpg

5B1G 5.jpg

5B1G 6.jpg

5B1G 7.JPG

5B1G 8.JPG

5B1G 9.JPG

5B1G 11.JPG

5B1G 13.JPG

5B1G 14.JPG

5B1G 15.JPG

5B1G 16.JPG

5B1G 17.JPG

5B1G 18.JPG

5B1G 19.JPG

5B1G 20.jpg

5B1G 21.JPG

5B1G 23.JPG

5B1G 24.JPG

5B1G 1.JPG

5B1G 2.jpg

5B1G 3.jpg

Web User Test 2

How We Conducted the UT

Objectives

  • To obtain feedback from our users with regards to the usability of the retail admin system
  • To create an application that is user-friendly for retailers

Details:

  • Participants: 10 users
  • Venue: SIS SR 2-4
  • Date: 25th Mar 2013 (Monday)
  • Duration: 4 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Web UT2 Instruction Sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use the retail admin system during the testing procedure.
  • Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for Retail Admin System

S/N Feature
1 Create a retail account
2 Login
3 Upload new E-coupon
4 Edit Pending E-coupon
5 Copy E-coupon
6 Delete E-coupon
7 View Published E-coupon statistics
8 Logout

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G WebUT2 Prob1.png

5B1G WebUT2 Prob2.png

Gallery

   

5B1G WebUT2 1.JPG

5B1G WebUT2 2.JPG

5B1G WebUT2 3.JPG

5B1G WebUT2 4.JPG

5B1G WebUT2 5.JPG

5B1G WebUT2 6.JPG

Web User Test 3

How We Conducted the UT

Objectives

  • To obtain feedback from our retailers with regards to the usability of the retailer admin system
  • To create an application that is user-friendly for retailers

Details:

  • Participants: 6 retailers
  • Venue: The Cathay
  • Date: 10th Apr 2013 (Wedneday)
  • Duration: 6 hours (each individual: 20-30mins)

Method

The retailer admin system was tested using a laptop . Each participant was given the Web UT3 Instruction Sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use the retail admin system during the testing procedure.
  • Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for Retail Admin System

S/N Feature
1 Create a retail account
2 Login
3 Upload new E-coupon
4 Edit Pending E-coupon
5 Copy E-coupon
6 Delete E-coupon
7 View Published E-coupon statistics
8 Delete E-coupon
9 Logout

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT 3 problem.png

User Test 6 at The Cathay

How We Conducted the UT

Objectives

We will be testing Count Me In android application at The Cathay and hope to achieve the following objectives:

  • To test and gather feedback on Count Me In in a deployed environment
  • To test the Cosine Similarity Index algorithm for the recommendation
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
  • To test the IPS with these test cases: UT6 Instructions Sheet

As the user test will be conducted in a deployed real life environment, no instructions sheet will be provided for the user for features that have already been tested. The user test is largely focused on the pushing of recommended E-coupons based on the user’s region.

Details:

  • Participants: 22 shoppers (18 undergraduates 4 public)
  • Venue: The Cathay
  • Participating Retail Outlets:
    • Popeyes
    • Createaf Waffles
    • Gong Cha
    • Marcella
    • St. Games
    • Jshoes
  • Date: 11th Apr 2013 (Thursday) & 12 Apr 2013 (Friday)
  • Duration: 4 hours each day (each individual: 20-30mins)

Method

As this is a field test taken under the conditions of actual operation, no instructions sheet was given to the users. Instead, the users were asked to download the application, redeem and E-coupon and then proceed to complete a survey giving their feedback on whether the app was easy to use and whether they would continue to use it. The only instructions that was given was pertaining to the indoor positioning system which had not been tested. For this field test, we invited students who belong to the targeted group of 'shoppers' and approached people from the public to download and use the application.

Data collection

Collecting of qualitative metrics

  • Conducted a survey of users who
  • Users will be asked to give remarks for each test case.

Features Tested (we tested those features that have not been tested)

S/N Feature
1 Find Retailer
1 Indoor Map

Results

Download the results here:

UT6 Results

Admin Web UT

How We Conducted the UT

Objectives

  • To obtain feedback from our with regards to the usability of the admin web portal
  • To create an application that is user-friendly for administrators

Details:

  • Participants: 6 retailers
  • Venue: The Cathay
  • Date: 16th Apr 2013 (Tuesday)
  • Duration: 6 hours (each individual: 20-30mins)

Method

The admin web system was tested using a laptop . Each participant was given the Admin Web Instruction Sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use the retail admin system during the testing procedure.
  • Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for Retail Admin System

S/N Feature
1 Login
2 View existing retailers
3 Delete existing retailer
4 Approving pending retailer
5 Toggle off and on IPS
6 Logout

Results

Download the results here:

Web User Test 4

How We Conducted the UT

Objectives

  • To obtain feedback from our retailers with regards to the usability of the retailer admin system with the updated changes from the last user test

Details:

  • Participants: 6 retailers
  • Venue: The Cathay
  • Date: 17th Apr 2013 (Wedneday)
  • Duration: 6 hours (each individual: 20-30mins)

Method

The retailer admin system was tested using a laptop . Each participant was given the Web UT4 Instruction Sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use the retail admin system during the testing procedure.
  • Upon completion of the web user test, the participant will be asked to give their feedback on the usability of the application.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for Retail Admin System

S/N Feature
1 Create a retail account
2 Login
3 Upload new E-coupon
4 Crop new E-coupon
5 Edit Pending E-coupon
6 Copy E-coupon
7 Delete E-coupon
8 View Published E-coupon statistics
9 View or export BI report
10 Delete E-coupon
11 Logout

Results

Download the results here: