HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T2 5 B 1 G Usability Documentation"

From IS480
Jump to navigation Jump to search
Line 61: Line 61:
 
|-
 
|-
 
|[[#User_Test_1| 1]]
 
|[[#User_Test_1| 1]]
|align="center"| Date: 2 Nov 2012 (Fri) <br> Participants: 7 shoppers/undergraduates <br>Location: SMU, SIS UPL Lab
+
|align="center"| Date: 2 Nov 2012 (Fri) <br> Participants: 7 shoppers <br>Location: SMU, SIS UPL Lab <br> Method: Paper Prototype
 
|align="center"| [[Media:5B1G_UT1_Execution_Plan.pdf|UT 1 Execution Plan]]
 
|align="center"| [[Media:5B1G_UT1_Execution_Plan.pdf|UT 1 Execution Plan]]
 
|align="center"| [[Media:5B1G_UT1_Results.pdf|UT 1 Results]]<br>
 
|align="center"| [[Media:5B1G_UT1_Results.pdf|UT 1 Results]]<br>
Line 67: Line 67:
 
|-
 
|-
 
|[[#User_Test_2| 2]]
 
|[[#User_Test_2| 2]]
|align="center"| Date: 14 Jan 2013 (Mon)
+
|align="center"| Date: 14 Jan 2013 (Mon) <br> Participants: 10 shoppers <br> Location: SMU, SIS UPL Lab <br> Method: Paper Prototype
 
|align="center"| [[Media:5B1G_UT2_Execution Plan.pdf|UT 2 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G_UT2_Execution Plan.pdf|UT 2 Execution Plan]] <br>
 
[[Media:5B1G_UT2_Test_sheet.xlsx|UT 2 Test Sheet]] <br>
 
[[Media:5B1G_UT2_Test_sheet.xlsx|UT 2 Test Sheet]] <br>
Line 74: Line 74:
 
|-
 
|-
 
|[[#User_Test_3| 3]]
 
|[[#User_Test_3| 3]]
|align="center"| Date: 15 Feb 2013 (Mon)
+
|align="center"| Date: 15 Feb 2013 (Mon) <br> Participants: 14 shoppers <br> Location: SMU SIS SR 3-4 <br> Method: Goals for app testing
 
|align="center"| [[Media:UT3_Execution_Plan.pdf|UT 3 Execution Plan]] <br>
 
|align="center"| [[Media:UT3_Execution_Plan.pdf|UT 3 Execution Plan]] <br>
 
[[Media:UT3_Instruction_sheet.pdf‎|UT 3 Instruction Sheet]] <br>
 
[[Media:UT3_Instruction_sheet.pdf‎|UT 3 Instruction Sheet]] <br>
Line 82: Line 82:
 
|-
 
|-
 
|[[#User_Test_4| 4]]
 
|[[#User_Test_4| 4]]
|align="center"| Date: 11 Mar 2013 (Mon)
+
|align="center"| Date: 11 Mar 2013 (Mon)  <br> Participants: 15 shoppers <br> Location:SR 2-3 (starting & ending point),GSR 3-5 (Pedro Shop), GSR 2-4 (The Basement)
 
|align="center"| [[Media:5B1G_UT4_Execution_Plan.pdf|UT 4 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G_UT4_Execution_Plan.pdf|UT 4 Execution Plan]] <br>
 
[[Media: UT4_Android_Instruction_sheet.pdf| UT4 Android Instructions Sheet]]<br>
 
[[Media: UT4_Android_Instruction_sheet.pdf| UT4 Android Instructions Sheet]]<br>
Line 89: Line 89:
 
|-
 
|-
 
|[[#User_Test_5| 5: SMU, The Basement Cafe]]
 
|[[#User_Test_5| 5: SMU, The Basement Cafe]]
|align="center"| Date: 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues)
+
|align="center"| Date: 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues) <br> Participants: 24 shoppers <br> Location: The Basement Café
 
|align="center"| [[Media:5B1G_UT5_with_The_Basement_Cafe.pdf|UT 5 Execution Plan]] <br>
 
|align="center"| [[Media:5B1G_UT5_with_The_Basement_Cafe.pdf|UT 5 Execution Plan]] <br>
 
[[Media: UT5_Instruction_sheet_(Facilitator).pdf| UT5 Android Instructions Sheet (Facilitator)]]<br>
 
[[Media: UT5_Instruction_sheet_(Facilitator).pdf| UT5 Android Instructions Sheet (Facilitator)]]<br>
Line 98: Line 98:
 
|-
 
|-
 
|[[#User_Test_6_.28The_Cathay.29| 6: The Cathay]]
 
|[[#User_Test_6_.28The_Cathay.29| 6: The Cathay]]
|align="center"| Date: 11 Apr 2013 (Thurs) & 12 Apr 2013 (Fri)
+
|align="center"| Date: 11 Apr 2013 (Thurs) & 12 Apr 2013 (Fri) <br>Participants: 22 shoppers <br> Location: The Basement Café
 
|align="center"| [[Media:UT6_at_The_Cathay.pdf|UT6 at The Cathay Execution Plan]] <br>
 
|align="center"| [[Media:UT6_at_The_Cathay.pdf|UT6 at The Cathay Execution Plan]] <br>
 
Link for Survey [http://smusg.asia.qualtrics.com/SE/?SID=SV_82OaL3JlHpOnttj here] <br>
 
Link for Survey [http://smusg.asia.qualtrics.com/SE/?SID=SV_82OaL3JlHpOnttj here] <br>
Line 199: Line 199:
  
 
====Change 1====
 
====Change 1====
<gallery widths="300" heights="200">
+
<gallery widths="400" heights="300">
 
Image: 5B1G_UT1_feedback1.png
 
Image: 5B1G_UT1_feedback1.png
 
Image: 5B1G_UT1_solution1.png
 
Image: 5B1G_UT1_solution1.png
Line 205: Line 205:
  
 
====Change 2====
 
====Change 2====
<gallery widths="300" heights="200">
+
<gallery widths="400" heights="300">
 
Image: 5B1G_UT1_feedback2.png
 
Image: 5B1G_UT1_feedback2.png
 
Image: 5B1G_UT1_solution2.png
 
Image: 5B1G_UT1_solution2.png
Line 211: Line 211:
  
 
====Change 3====
 
====Change 3====
<gallery widths="300" heights="200">
+
<gallery widths="400" heights="300">
 
Image: 5B1G_UT1_feedback3.png
 
Image: 5B1G_UT1_feedback3.png
 
Image: 5B1G_UT1_solution3.png
 
Image: 5B1G_UT1_solution3.png
Line 217: Line 217:
  
 
====Change 4====
 
====Change 4====
<gallery widths="300" heights="200">
+
<gallery widths="400" heights="300">
 
Image: 5B1G_UT1_feedback4.png
 
Image: 5B1G_UT1_feedback4.png
 
Image: 5B1G_UT1_solution4.png
 
Image: 5B1G_UT1_solution4.png
Line 223: Line 223:
  
 
====Change 5====
 
====Change 5====
<gallery widths="300" heights="200">
+
<gallery widths="400" heights="300">
 
Image: 5B1G_UT1_feedback5.png
 
Image: 5B1G_UT1_feedback5.png
 
Image: 5B1G_UT1_solution5.png
 
Image: 5B1G_UT1_solution5.png

Revision as of 19:19, 19 April 2013

5B1G Logo.png


Home

Our Team/Sponsors

Project Overview

Project Management

Project Documentation

Usability Documentation

Learning Outcomes


Count Me In User Tests

Webapp User Tests

Prototypes & Mockups

Summary of User Tests Documentation
User Test Details Documents for User Test User Test Results
1 Date: 2 Nov 2012 (Fri)
Participants: 7 shoppers
Location: SMU, SIS UPL Lab
Method: Paper Prototype
UT 1 Execution Plan UT 1 Results

UT 1 Video

2 Date: 14 Jan 2013 (Mon)
Participants: 10 shoppers
Location: SMU, SIS UPL Lab
Method: Paper Prototype
UT 2 Execution Plan

UT 2 Test Sheet
UT 2 User Appreciation

UT 2 Results
3 Date: 15 Feb 2013 (Mon)
Participants: 14 shoppers
Location: SMU SIS SR 3-4
Method: Goals for app testing
UT 3 Execution Plan

UT 3 Instruction Sheet
Link for Survey here

UT 3 Raw Qualitative Results

UT 3 Compiled Results

4 Date: 11 Mar 2013 (Mon)
Participants: 15 shoppers
Location:SR 2-3 (starting & ending point),GSR 3-5 (Pedro Shop), GSR 2-4 (The Basement)
UT 4 Execution Plan

UT4 Android Instructions Sheet

UT 4 Raw Qualitative Results

UT 4 Compiled Results

5: SMU, The Basement Cafe Date: 18 Mar 2013 (Mon) & 19 Mar 2013 (Tues)
Participants: 24 shoppers
Location: The Basement Café
UT 5 Execution Plan

UT5 Android Instructions Sheet (Facilitator)
UT5 Instructions Sheet (User)
Link for Survey here

UT 5 Raw Qualitative Results

UT 5 Compiled Results

6: The Cathay Date: 11 Apr 2013 (Thurs) & 12 Apr 2013 (Fri)
Participants: 22 shoppers
Location: The Basement Café
UT6 at The Cathay Execution Plan

Link for Survey here
IPS Instructions Sheet

UT6 Results

Test Plan
Download the test plan here

Compatibility Information (NFC enabled device):
Samsung S3, Samsung Galaxy Note 2
Samsung S2*, HTC One*
*These devices are unable to carry out the NFC redemption

Our application is able to run on Ice Cream Sandwich (API level 15) onwards

Incompatible with the following device:
HTC Desire | Froyo (API level 8)
MotoBlur | Froyo (API level 8)

Contents

User Test 1

How We Conducted the UT

Objectives

To understand whether the user finds the usability of the application user-friendly The table below shows a list of features that will tested for our paper prototype user test to test the usability of the new layout

Details:

  • Participants: 7 shoppers/ undergraduates
  • Venue: SMU, SIS UPL Lab
  • Date: 2nd Nov 2012 (Friday)
  • Duration: 1 hour

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.

Features Tested

S/N Feature
1 Sign in as an existing user
2 Login with Facebook
3 Bring me there
4 Share me (share on facebook)
5 Bookmark me
6 Scan a product and add to cart
7 View my shopping cart/checkout
8 View past transactions
9 Browse coupons
10 View recommendations

Results

Observations

95%

Feedback

95%

Gallery

Videos:
UT 1 Video
5B1G UT 1.PNG

Changes Made

Change 1

Change 2

Change 3

Change 4

Change 5

User Test 2

How We Conducted the UT

Objectives

The main objectives of this user test are:

  • To obtain feedback from our users with regards to the layout in our application, and
  • To implement a layout that is considered to be of acceptable standard by users

Details:

  • Participants: 10 shoppers/undergraduates
  • Venue: SMU, SIS UPL Lab
  • Date: 14th Jan 2013
  • Duration: 2 hours

Method

We conducted the user test using a low-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure.
  • Upon completion of the paper prototype, the participant will be asked to give their feedback on usability and design of the application.
  • We will compile the problems and classify them using a set of heuristics, thereafter we will assign a severity rating to each compiled problem

Features Tested

S/N Feature
1 Login with facebook
2 Push notification, view recommended coupons
3 View home page/featured coupons
4 Share me (share on facebook)
5 Bring me there (navigation)
6 Add to Coupon Cart
7 Use coupon (NFC tag)
8 Browse coupon (by category & location)
9 Map
10 Coupon Cart
11 Logout of application

Results

Detailed Results

Download here: UT 2 Results

Compiled Problems & Proposed Solutions

5B1G UT2 Problem1.png

5B1G UT2 Problem2.png

5B1G UT2 Problem3.png

5B1G UT2 Problem4.png

5B1G UT2 Problem5.png


Gallery

   

5B1G UT2 4.jpg

5B1G UT2 5.jpg

5B1G UT2 6.jpg

5B1G UT2 1.png

5B1G UT2 2.png

5B1G UT2 3.png

Changes Made (during the UT)

Initial paper prototype Changes made during UT2
5B1G arrow paper prototype.jpg

Most participants failed to realize that the arrow on the top left corner was clickable

5B1G Arrow changes.JPG

Changed the icon to a more obvious and clickable icon that does not that does
that does not look like part of the app logo


User Test 3

How We Conducted the UT

Objectives

  • To test the functionalities of the application from a user's perspective
  • To obtain feedback from our users with regards to the usability of the application

Details:

  • Participants: 14 shoppers/ undergraduates
  • Venue: SMU, SIS SR 3-4
  • Date: 15th Feb 2013 (Friday)
  • Duration: 3 hours

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the instruction sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe the how quickly the participant is able to complete the test case and whether he or she meets with any confusion (such as clicking on the wrong item) during each test case. These observations will be recorded down

Collecting of quantitative metrics

  • Upon completion of each test case, the participant will be asked to rate how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested

S/N Feature
1 Login with Facebook
2 View Home Paage/ Latest e-coupons
3 Share Me (share on Facebook)
4 Coupon Cart
5 Use Me (NFC tag)
6 Search Coupon (search by category and location
7 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT3 Problem1.png

5B1G UT3 Problem2.png

5B1G UT3 Problem3.png

5B1G UT3 Problem4.png

5B1G UT3 Problem5.png

Gallery

   

5B1G UT3 1.JPG

5B1G UT3 2.JPG

5B1G UT3 3.JPG

5B1G UT3 4.JPG

5B1G UT3 5.jpg

5B1G UT3 6.JPG

5B1G UT3 7.jpg

5B1G UT3 8.JPG

5B1G UT3 9.JPG

5B1G UT3 13.jpg

5B1G UT3 10.JPG

5B1G UT3 11.JPG

5B1G UT3 14.JPG

5B1G UT3 16.JPG

5B1G UT3 18.JPG

User Test 4

How We Conducted the UT

Objectives

  • To test the functionalities of the both the android application and the web application in a simulated real life environment
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept

Details:

  • Participants: 15 shoppers/ undergraduates
  • Venue:
    • Testing venues for Count Me In android app:
      • SR 2-3 (starting & ending point)
      • GSR 3-5 (Pedro Shop)
      • GSR 2-4 (The Basement)
  • Date: 11th Mar 2013 (Monday)
  • Duration: 4 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
  • Users will be asked to give remarks for each test case.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for “Count Me In”

S/N Feature
1 Login with Facebook
2 View Latest E-coupons
3 Share Me
4 Add to Favourites
5 Push notification to view recommended E-coupons
6 Indoor Navigation
7 Receive location-specific push notification & view recommended e-coupons
8 Use Me
9 Bring Me
10 Delete E-coupon
11 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT4 Problem1.png

5B1G UT4 Problem2.png

Gallery

   

5B1G UT4 1.jpg

5B1G UT4 2.JPG

5B1G UT4 3.JPG

5B1G UT4 10.JPG

5B1G UT4 8.JPG

5B1G UT4 9.JPG

User Test 5

How We Conducted the UT

Objectives

  • To test the functionalities of the android application in a real life environment
  • To stress-test the application in a condition where many users will be using the application
  • To gather feedback from our users with regards to the functionalities of the application and

the process flow of the concept

Details:

  • Participants: 24 shoppers/ undergraduates
  • Venue: The Basement Café
  • Date: 18th Mar 2013 (Monday), 19th Mar 2013 (Tuesday)
  • Duration: 4.5 hours (each individual: 20-30mins)

Method

The actual application was tested this time using the android device, Samsung Galaxy S III. Each participant was given the Android instructions sheet and asked to circle how easy it was to complete each test case.

Data collection

Collecting of qualitative metrics

  • Facilitators will observe how the participants use both the web and mobile application during the testing procedure.
  • Users will be asked to give remarks for each test case.

Collecting of quantitative metrics

  • Upon completion of the UT, the participant will be asked to rate, on a scale of 1 to 5, how easy the task is and to give their feedback on the functionality and the usability of the application.

Features Tested for “Count Me In”

S/N Feature
1 Login with Facebook
2 View Latest E-coupons
3 Indoor Navigation
4 Bring Me
5 Receive location-specific push notification & view Recommended
6 Share Me
7 Favourite Me
8 View Categories
9 Delete E-coupons
10 Use Me
11 Logout of application

Results

Download the results here:

Compiled Problems & Proposed Solutions

5B1G UT5 Problem1.png

Gallery

   

5B1G 4.jpg

5B1G 5.jpg

5B1G 6.jpg

5B1G 7.JPG

5B1G 8.JPG

5B1G 9.JPG

5B1G 11.JPG

5B1G 13.JPG

5B1G 14.JPG

5B1G 15.JPG

5B1G 16.JPG

5B1G 17.JPG

5B1G 18.JPG

5B1G 19.JPG

5B1G 20.jpg

5B1G 21.JPG

5B1G 23.JPG

5B1G 24.JPG

5B1G 1.JPG

5B1G 2.jpg

5B1G 3.jpg



User Test 6 (The Cathay)

How We Conducted the UT

Objectives

We will be testing Count Me In android application at The Cathay and hope to achieve the following objectives:

  • To test and gather feedback on Count Me In in a deployed environment
  • To test the Cosine Similarity Index algorithm for the recommendation
  • To gather feedback from our users with regards to the functionalities of the application and the process flow of the concept
  • To test the IPS with these test cases: UT6 Instructions Sheet

As the user test will be conducted in a deployed real life environment, no instructions sheet will be provided for the user for features that have already been tested. The user test is largely focused on the pushing of recommended E-coupons based on the user’s region.

Details:

  • Participants: 22 shoppers (18 undergraduates 4 public)
  • Venue: The Cathay
  • Participating Retail Outlets:
    • Popeyes
    • Createaf Waffles
    • Gong Cha
    • Marcella
    • St. Games
    • Jshoes
  • Date: 11th Apr 2013 (Thursday) & 12 Apr 2013 (Friday)
  • Duration: 4 hours each day (each individual: 20-30mins)

Method

As this is a field test taken under the conditions of actual operation, no instructions sheet was given to the users. Instead, the users were asked to download the application, redeem and E-coupon and then proceed to complete a survey giving their feedback on whether the app was easy to use and whether they would continue to use it. The only instructions that was given was pertaining to the indoor positioning system which had not been tested. For this field test, we invited students who belong to the targeted group of 'shoppers' and approached people from the public to download and use the application.

Data collection

Collecting of qualitative metrics

  • Conducted a survey of users who
  • Users will be asked to give remarks for each test case.

Features Tested (we tested those features that have not been tested)

S/N Feature
1 Find Retailer
1 Indoor Map

Results

Download the results here:

UT6 Results

Gallery

   

5B1G UT6 1.jpg

5B1G UT6 5.jpg

5B1G UT6 3.jpg

5B1G UT6 4.jpg

5B1G UT6 2.jpg

5B1G UT6 6.jpg

5B1G UT6 7.jpg

5B1G UT6 8.jpg

5B1G UT6 9.jpg