HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T2 Can.One Testing"

From IS480
Jump to navigation Jump to search
 
(13 intermediate revisions by the same user not shown)
Line 14: Line 14:
 
| style="padding:0 .3em;  font-size:100%; border-right:2px solid #800000; text-align:center; background-color:#FAFAD2; " width="16.67%" | [[IS480_Team_wiki:_2012T2_Can.One_Testing|<font face = "century gothic" color="#800000">Testing</font>]]
 
| style="padding:0 .3em;  font-size:100%; border-right:2px solid #800000; text-align:center; background-color:#FAFAD2; " width="16.67%" | [[IS480_Team_wiki:_2012T2_Can.One_Testing|<font face = "century gothic" color="#800000">Testing</font>]]
 
| style="padding:0 .3em;  font-size:100%; text-align:center; background-color:#ffffff; " width="16.67%" | [[IS480_Team_wiki:_2012T2_Can.One_Learning_Outcome|<font face = "century gothic" color="#800000">Learning Outcome</font>]]
 
| style="padding:0 .3em;  font-size:100%; text-align:center; background-color:#ffffff; " width="16.67%" | [[IS480_Team_wiki:_2012T2_Can.One_Learning_Outcome|<font face = "century gothic" color="#800000">Learning Outcome</font>]]
 +
|}
 +
 +
<!--Sub-menu-->
 +
{| style="background-color:#4c4942; color:#504A4B padding: 5px 0 0 0;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0" |
 +
| style="padding:0.3em; font-size:90%; background-color:#FAFAD2; border-bottom:2px solid #504A4B; text-align:center; " width="20%"|[[IS480_Team_wiki:_2012T2_Can.One_Testing#User_Testing_1 |<font face = "century gothic" color="#800000">User Testing 1</font>]]
 +
| style="padding:0.3em; font-size:90%; background-color:#FAFAD2; border-bottom:2px solid #504A4B; text-align:center; " width="20%"|[[IS480_Team_wiki:_2012T2_Can.One_Testing#User_Testing_2 |<font face = "century gothic" color="#800000">User Testing 2</font>]]
 +
| style="padding:0.3em; font-size:90%; background-color:#FAFAD2; border-bottom:2px solid #504A4B; text-align:center; " width="20%"|[[IS480_Team_wiki:_2012T2_Can.One_Testing#User_Testing_3 |<font face = "century gothic" color="#800000">User Testing 3</font>]]
 +
| style="padding:0.3em; font-size:90%; background-color:#FAFAD2; border-bottom:2px solid #504A4B; text-align:center; " width="20%"|[[IS480_Team_wiki:_2012T2_Can.One_Testing#User_Acceptance_Test |<font face = "century gothic" color="#800000">User Acceptance Test</font>]]
 
|}
 
|}
  
 
<font face="century gothic" size="2">
 
<font face="century gothic" size="2">
 
+
__NOTOC__
===<div style="color: #ffffff; font-size:0%">Usability Testing 1</div>===
+
===<div style="color: #ffffff; font-size:0%">User Testing 1</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>Usability Testing 1</b>
+
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>User Testing 1</b>
 
|-
 
|-
  
Line 28: Line 36:
 
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
<!--PUT CONTENT HERE-->
 
<!--PUT CONTENT HERE-->
'''Date:''' Mid January <br>
+
'''Date:''' 18-19 January 2013 <br>
 +
<br>
 +
'''Location:''' SIS CR 3.2 (Day 1) & SIS SR 3.2 (Day 2)
 +
<br>
 
<br>
 
<br>
 
'''Objective:'''   
 
'''Objective:'''   
Line 34: Line 45:
 
#Collect feedbacks from users regarding application’s usability
 
#Collect feedbacks from users regarding application’s usability
 
<br>
 
<br>
'''Scope:'''   
+
'''Scope Tested:'''   
 
*Registration & Login
 
*Registration & Login
*Create & Display Avatar
+
*Avatar Creation
*Add Accessories to 3D Avatar
+
*Shopping
*Profile Page
+
**Try-on Product
*Shopping
+
**Add Item to Cart
**Shopping Cart
+
**Check Out
 
**Payment Gateway
 
**Payment Gateway
**Promotions
+
*Search Friend
**Rewards
+
 
*Social
 
**Friends
 
**Chat
 
**Groups
 
 
<br>
 
<br>
'''Testers:'''  People who match VEON’s target user profile (young adults, age 15-25)<br>
+
'''Testers:'''  30 people (young adults, age 15-30)<br>
 +
 
 
<br>
 
<br>
 
<ins>'''Testing Methodology'''</ins><br>
 
<ins>'''Testing Methodology'''</ins><br>
Testers will be given a task list that will involve completing activities based on the tested functionalities. They are encouraged to think aloud, while facilitators are there to help and observe.<br>
+
*5 testers will concurrently test the application in the chosen time slot (@ ±30 minutes)
 +
*When arrived, testers will need to sign a consent form and complete a pre-test questionnaire
 +
*Testers will be given a task list to be completed. For each task, they need to indicate the completion status (pass/fail), the difficulty level (low, medium, high), and comment for the tested functionality.
 +
*Testers are encouraged to think out loud. Each tester will be closely observed by an observer, who will take notes of tester’s behavior and comment.
 +
*Upon completing the test, testers will need to complete a post-test questionnaire and subsequently get 2 Subway cookies as a reward
 +
<br>
 +
We need to ensure that at least 75% of the testers (22 people) will come again for UAT 1. Therefore during UT 1, testers will be given [$2 worth of item] and a $5 voucher that can be redeemed if they come for UAT 1<br>
 
<br>
 
<br>
 
During the test, we will collect the following '''metrics''':
 
During the test, we will collect the following '''metrics''':
 
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
 
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
#Time spent on a task - Using a time clicker, we record the completion time of each task.
+
#Perceived difficulties of each task
#Task or test level satisfaction rate - After completion of each task, testers are to fill an online survey that will record their demographics, as well as their feedbacks on each task and the application in general.  
+
#Comments regarding any features or the application
 +
<br>
 +
<br>
 +
'''Documents:'''
 +
*[http://dl.dropbox.com/u/18512978/FYP/UT%201/Consent%20and%20Waiver.doc Consent Letter]
 +
*[http://dl.dropbox.com/u/18512978/FYP/UT%201/Pre-Test%20Questionnaire.doc Pre-Test Questionnaire]
 +
*[http://dl.dropbox.com/u/18512978/FYP/UT%201/UT1%20-%20Test%20Cases.xls Test Cases]
 +
*[http://dl.dropbox.com/u/18512978/FYP/UT%201/Post-Test%20Questionnaire.doc Post-Test Questionnaire]
 
<br>
 
<br>
 
|-
 
|-
Line 65: Line 86:
 
<div style = "text-align:right;"><font ="color:#800000; size:"12px;"><u>[[IS480_Team_wiki:_2012T2_Can.One_Testing#Top|Back to Top]]</u></font></div>
 
<div style = "text-align:right;"><font ="color:#800000; size:"12px;"><u>[[IS480_Team_wiki:_2012T2_Can.One_Testing#Top|Back to Top]]</u></font></div>
  
===<div style="color: #ffffff; font-size:0%">Usability Testing 2</div>===
+
===<div style="color: #ffffff; font-size:0%">User Testing 2</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>Usability Testing 2</b>
+
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>User Testing 2</b>
 
|-
 
|-
  
Line 75: Line 96:
 
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
<!--PUT CONTENT HERE-->
 
<!--PUT CONTENT HERE-->
'''Date:''' Beginning of March <br>
+
'''Date:''' 7-8 March 2013 <br>
 +
<br>
 +
'''Location:''' SOB SR 2.2 (Day 1) & SIS CR 3.1 (Day 2)
 +
<br>
 
<br>
 
<br>
 
'''Objective:'''   
 
'''Objective:'''   
Line 81: Line 105:
 
#Collect feedbacks from users regarding application’s usability
 
#Collect feedbacks from users regarding application’s usability
 
<br>
 
<br>
'''Scope:''' TBA <br>
+
'''Scope Tested:'''
 +
* User registration
 +
* Style and preferences
 +
* Try-on product
 +
* Shopping process
 +
* Past purchases
 +
* Social features
 +
 
 
<br>
 
<br>
'''Testers:'''  People who match VEON’s target user profile (young adults, age 15-25)<br>
+
'''Testers:'''  40 people (young adults, age 19-26)<br>
 +
 
 
<br>
 
<br>
 
<ins>'''Testing Methodology'''</ins><br>
 
<ins>'''Testing Methodology'''</ins><br>
Testers will be given a task list that will involve completing activities based on the tested functionalities. They are encouraged to think aloud, while facilitators are there to help and observe.<br>
+
* Testers were asked to fill in registration form to indicate their availability
 +
* When arrived, testers were provided with test scenario, and they were asked to roam around the web application accordingly. They were also encouraged to think out loud since each tester will be closely observed by an observer, who will take notes of tester’s behavior and comment.
 +
* After completing the scenario, testers were asked to fill in evaluation sheet to indicate the difficulty level of particular task (scale 1-10) and comment.
 +
* Upon completing the test, testers will get a delicious Subway sandwich as a reward
 
<br>
 
<br>
 
During the test, we will collect the following '''metrics''':
 
During the test, we will collect the following '''metrics''':
 
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
 
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
#Time spent on a task - Using a time clicker, we record the completion time of each task.
+
#Perceived difficulties of each task
#Task or test level satisfaction rate - After completion of each task, testers are to fill an online survey that will record their demographics, as well as their feedbacks on each task and the application in general.  
+
#Comments regarding any features or the application
 +
<br>
 +
<br>
 +
'''Documents:'''
 +
*[http://dl.dropboxusercontent.com/u/18512978/FYP/UT%202/UT2%20-%20Test%20Cases.docx UT2 Test Scenario]
 
<br>
 
<br>
 
|-
 
|-
Line 99: Line 138:
 
<div style = "text-align:right;"><font ="color:#800000; size:"12px;"><u>[[IS480_Team_wiki:_2012T2_Can.One_Testing#Top|Back to Top]]</u></font></div>
 
<div style = "text-align:right;"><font ="color:#800000; size:"12px;"><u>[[IS480_Team_wiki:_2012T2_Can.One_Testing#Top|Back to Top]]</u></font></div>
  
===<div style="color: #ffffff; font-size:0%">Users Acceptance Test</div>===
+
===<div style="color: #ffffff; font-size:0%">User Testing 3</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>Users Acceptance Test</b>
+
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>User Testing 3</b>
 
|-
 
|-
  
Line 109: Line 148:
 
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
<!--PUT CONTENT HERE-->
 
<!--PUT CONTENT HERE-->
'''Date:''' Beginning of April <br>
+
'''Date:''' 5 April 2013 <br>
 
<br>
 
<br>
'''Objective:''' Determine if users find our application intuitive enough, and can navigate without instructions.<br>
+
'''Objective:'''
 +
#Observe how users interact with our application
 +
#Collect feedbacks from users regarding application’s usability
 
<br>
 
<br>
'''Scope:''' TBA <br>
+
'''Scope Tested:''' ALL <br>
 
<br>
 
<br>
'''Testers:'''   
+
'''Testers:'''  15 people (NON SMU students, age 22-28)<br>
*Gamurai
+
 
*People who match VEON’s target user profile (young adults, age 15-25)
+
<br>
 +
<ins>'''Testing Methodology'''</ins><br>
 +
* Testers were asked to fill in registration form to indicate their availability
 +
* When arrived, testers were provided with test scenario, and they were asked to roam around the web application accordingly. They were also encouraged to think out loud since each tester will be closely observed by an observer, who will take notes of tester’s behavior and comment.
 +
* After completing the scenario, testers were asked to fill in evaluation sheet to indicate the difficulty level of particular task (scale 1-10) and comment.
 +
* Upon completing the test, testers will get a delicious Subway sandwich as a reward
 +
<br>
 +
During the test, we will collect the following '''metrics''':
 +
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
 +
#Perceived difficulties of each task
 +
#Comments regarding any features or the application
 +
<br>
 +
<br>
 +
'''Documents:'''
 +
*[http://dl.dropboxusercontent.com/u/18512978/FYP/UT%203/UT3%20-%20Test%20Cases%20and%20Scenario.docx UT3 Test Scenario]
 +
<br>
 +
|-
 +
| style= "background:#800000; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
|-
 +
|}
 +
<div style = "text-align:right;"><font ="color:#800000; size:"12px;"><u>[[IS480_Team_wiki:_2012T2_Can.One_Testing#Top|Back to Top]]</u></font></div>
 +
 
 +
===<div style="color: #ffffff; font-size:0%">User Acceptance Test</div>===
 +
{| class="wikitable" cellpadding="15"
 +
|-
 +
| style="background:#ffffff;; color: #800000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:left; border: 1px solid #ffffff" valign="top" | <b>User Acceptance Test</b>
 +
|-
 +
 
 +
| style= "background:#800000; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
|-
 +
| style= "background:#FAFAD2; color: #800000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
<!--PUT CONTENT HERE-->
 +
'''Date:''' 15 April 2013 <br>
 +
<br>
 +
'''Objective:'''
 +
* To obtain confirmation that the application meets the mutually agreed-upon requirements
 +
* Collect feedbacks from sponsors regarding application’s usability and performance
 +
<br>
 +
<br>
 +
'''Scope:'''
 +
* User management
 +
* 3D visualization
 +
* Try-on product
 +
* Shopping process
 +
* Social features
 +
* Recommendation
 +
<br>
 +
* Vendor portal
 +
<br>
 +
<br>
 +
'''Testers:''' Gamurai
 
<br>
 
<br>
 
<ins>'''Testing Methodology'''</ins><br>
 
<ins>'''Testing Methodology'''</ins><br>
Line 124: Line 215:
 
During the test, we will collect the following '''metrics''':
 
During the test, we will collect the following '''metrics''':
 
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
 
#Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
#Time spent on a task - Using a time clicker, we record the completion time of each task.
+
#Task or test level satisfaction rate - After completion of each task, testers are to fill a post-survey that will record their feedbacks on each task and the application in general.  
#Task or test level satisfaction rate - After completion of each task, testers are to fill an online survey that will record their demographics, as well as their feedbacks on each task and the application in general.  
+
<br>
 +
'''Documents:'''
 +
*[http://dl.dropboxusercontent.com/u/18512978/FYP/UAT%201/UAT%201%20-%20Test%20Scenario%20%26%20Cases%20.docx UAT Test Scenario]
 
<br>
 
<br>
 
|-
 
|-

Latest revision as of 23:24, 14 April 2013

Home Project Overview Project Management Project Documentation Testing Learning Outcome
User Testing 1 User Testing 2 User Testing 3 User Acceptance Test

User Testing 1

User Testing 1

Date: 18-19 January 2013

Location: SIS CR 3.2 (Day 1) & SIS SR 3.2 (Day 2)

Objective:

  1. Observe how users interact with our application
  2. Collect feedbacks from users regarding application’s usability


Scope Tested:

  • Registration & Login
  • Avatar Creation
  • Shopping
    • Try-on Product
    • Add Item to Cart
    • Check Out
    • Payment Gateway
  • Search Friend


Testers: 30 people (young adults, age 15-30)


Testing Methodology

  • 5 testers will concurrently test the application in the chosen time slot (@ ±30 minutes)
  • When arrived, testers will need to sign a consent form and complete a pre-test questionnaire
  • Testers will be given a task list to be completed. For each task, they need to indicate the completion status (pass/fail), the difficulty level (low, medium, high), and comment for the tested functionality.
  • Testers are encouraged to think out loud. Each tester will be closely observed by an observer, who will take notes of tester’s behavior and comment.
  • Upon completing the test, testers will need to complete a post-test questionnaire and subsequently get 2 Subway cookies as a reward


We need to ensure that at least 75% of the testers (22 people) will come again for UAT 1. Therefore during UT 1, testers will be given [$2 worth of item] and a $5 voucher that can be redeemed if they come for UAT 1

During the test, we will collect the following metrics:

  1. Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
  2. Perceived difficulties of each task
  3. Comments regarding any features or the application



Documents:


User Testing 2

User Testing 2

Date: 7-8 March 2013

Location: SOB SR 2.2 (Day 1) & SIS CR 3.1 (Day 2)

Objective:

  1. Observe how users interact with our application
  2. Collect feedbacks from users regarding application’s usability


Scope Tested:

  • User registration
  • Style and preferences
  • Try-on product
  • Shopping process
  • Past purchases
  • Social features


Testers: 40 people (young adults, age 19-26)


Testing Methodology

  • Testers were asked to fill in registration form to indicate their availability
  • When arrived, testers were provided with test scenario, and they were asked to roam around the web application accordingly. They were also encouraged to think out loud since each tester will be closely observed by an observer, who will take notes of tester’s behavior and comment.
  • After completing the scenario, testers were asked to fill in evaluation sheet to indicate the difficulty level of particular task (scale 1-10) and comment.
  • Upon completing the test, testers will get a delicious Subway sandwich as a reward


During the test, we will collect the following metrics:

  1. Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
  2. Perceived difficulties of each task
  3. Comments regarding any features or the application



Documents:


User Testing 3

User Testing 3

Date: 5 April 2013

Objective:

  1. Observe how users interact with our application
  2. Collect feedbacks from users regarding application’s usability


Scope Tested: ALL

Testers: 15 people (NON SMU students, age 22-28)


Testing Methodology

  • Testers were asked to fill in registration form to indicate their availability
  • When arrived, testers were provided with test scenario, and they were asked to roam around the web application accordingly. They were also encouraged to think out loud since each tester will be closely observed by an observer, who will take notes of tester’s behavior and comment.
  • After completing the scenario, testers were asked to fill in evaluation sheet to indicate the difficulty level of particular task (scale 1-10) and comment.
  • Upon completing the test, testers will get a delicious Subway sandwich as a reward


During the test, we will collect the following metrics:

  1. Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
  2. Perceived difficulties of each task
  3. Comments regarding any features or the application



Documents:


User Acceptance Test

User Acceptance Test

Date: 15 April 2013

Objective:

  • To obtain confirmation that the application meets the mutually agreed-upon requirements
  • Collect feedbacks from sponsors regarding application’s usability and performance



Scope:

  • User management
  • 3D visualization
  • Try-on product
  • Shopping process
  • Social features
  • Recommendation


  • Vendor portal



Testers: Gamurai
Testing Methodology
Testers will be given a task list that will involve completing activities based on the tested functionalities. Facilitators will only observe, no guidance and instruction steps will be provided.

During the test, we will collect the following metrics:

  1. Error rate - Using screen recording software, we will see if a task is completed with ease or if a task is only completed after many wrong turns.
  2. Task or test level satisfaction rate - After completion of each task, testers are to fill a post-survey that will record their feedbacks on each task and the application in general.


Documents: