HeaderSIS.jpg

2011T2 Bazinga UserTest2

From IS480
Jump to navigation Jump to search
Main Wiki Page   Project Management   Project Metrics   Technical Complexity & Deployment   User Test 2   Beta Test   BP Tech Fair   Team Reflections


Bazinga user test2.png







UT 2 Summary

Our User Test 2 took place on 21/3/12and ended on 28/3/12.
A total of 35 testers participated.
In order to ensure that our test results are not affected by any biasness, our testers included students from different faculties and include a mixture of both non-Kinect and Kinect users so that we can anticipate the general public's receptiveness to our application once it is launched. The following documents were what we prepared for our User Testing 2.

Feel free to view the schedule and User Testing execution process that has been prepared, as shown in the first external link. The second link is the feedback form that all testers will fill in after trying out the testing.


Prepared Documents Link
User Test 2 Schedule, Execution Process UT 2 Schedule and Execution process
User Test 2 Feedback Questions SMU Quadtrics Feedback Questions
Download UT 2 Feedback
User Test 2 Plan Download UT 2 Plan
User Test 2 Video Tutorial Transformation Process Our Video Tutorial Improvements and Timeline
User Test 2 Instructions for Testers Download instructions
UAT Results/Analysis UT 2 Feedback Form Results
User Test 2 Schedule and Final Results UT 2 Observation Results

UT 2 Setup


Introduction

After our MidTerm presentation, we decided to improve our User Testing process, especially since we are creating an application on the Kinect platform, which is still a new technology in the industry as compared to mobile phones. Thus, we decided to implement black box testing, which will enable us to get both qualitative and quantitative data from this User Test 2. This will enable us to understand what are the main issues faced by the testers when placed in a quiet environment where they will not be able to ask the developers any questions pertaining to the list of tasks they will have to complete.

UT2 Goals

Collect useful usability and gesture feedback from our testers to better understand their needs and improve our existing application.

UT2 Objectives

  • Unguided Test for all testers
  • Verify if the user finds it intuitive and easy to pick up hand gestures
  • Verify the usability and easy navigation of the application
  • Gather user opinion on Kinect Application feasibility in a retail setting

Room setup

Bazinga setup.png

Equipment Used

Bazinga equipment.png

Observers’ Tasks

Observer's points to note for each Tester Observer's procedures
Here are a list of points that we have to do to ensure that we captured all the relevant data from the tester.
  • Time to perform a task
  • Percentage of task completed
  • Number of errors
  • Number of repetitions or failed commands
  • Number of features or commands not used
  • Time spent navigating and/or searching for information
  • Number of clicks or taps to perform a task
  • Quality/Quantity of information found
Collocate log files for each participant/scenario

Summarize each scenario across all participants, includes plots and trend analysis

  • Quantitative measures
  • Qualitative observations
  • Questionnaire answers

Make conclusions for each test goal



Difference from UT 1


UT 1 and 2 Differences
Bazinga ut 1 2 difference.png



Video Tutorial Improvements and Timeline

Please go to our Video Tutorial Improvements and Timeline which will show you 6 different versions over the past few weeks.

UT 2 Process

Black Box Testing

To ensure that our 2T1 meets our objectives in allowing the tester to use the application without our help, we made it a non guided User Test. We wanted to verify if the users could pick up the gestures naturally and provide us opinion on how our group can further improve our application. Thus, we will place instructions at the side of the tester (both left and right) and the user will follow the steps required to allow us to know which features require more help from the user, without the need of someone to provide immediate help.

Bazinga testing.png


Please go to our Video Tutorial Improvements and Timeline which will show you 6 different versions over the past few weeks.


UT 2 Results


Testers Demographics

UT2 breakdown.png


Type Number Percentage
SIS Students 25 71%
Non-IS Students 9 26%
Professors and Instructors 1 3%

Comments: Non-SIS students represent the user population from other faulties in SMU and their comments tend to be much less technical, reflecting most of the actual target public users. In addition, they also require much more guidance than SIS students, who are more experienced in application development and new technologies. Non SIS students provided us with an interesting dimension to the feedback collected, such as more of the business feasibility of the Kinect application, as compared to the SIS Students who will be more inclined towards the technology, and will tell more of which functions are more useful.

The instructor, on the other hand, is also very technical and provided us the best advice and opinions for a user to pick up gestures naturally, and how the application will be suitable for the public, based onher vast experiences with lots of applications.



In addition, we had a good mix of males and females, giving us a good mix of reviews, not bias only to one gender.


UT2Testers.png

Type Number Percentage
Male 19 54%
Female 16 46%




User Testing 2 Outcome

Apart from quantitative results, our team has received very positive feedbacks and suggestions on how we can improve our application. Most of the testers feel that the Kinect application is cool, interactive, and still room to improve in the Augmented Reality function and the gestures used in the application. They see a potential in this application but a minority of 3 testers feel that they do not want to use the application again. All other testers will want to use our application again.


User Testing 2 Documents

For more information on our UAT 1, you can download our documents below:

Documents Link
UAT Results/Analysis UT 2 Feedback Form Results
User Test 2 Schedule and Final Results UT 2 Observation Results