2011T2 Bazinga UserTest2
Main Wiki Page | Project Management | Project Metrics | Technical Complexity & Deployment | User Test 2 | Beta Test | BP Tech Fair | Team Reflections |
Contents
UT 2 Summary
Our User Test 2 took place on 21/3/12and ended on 28/3/12.
A total of 35 testers participated.
In order to ensure that our test results are not affected by any biasness, our testers included students from different faculties and include a mixture of both non-Kinect and Kinect users so that we can anticipate the general public's receptiveness to our application once it is launched. The following documents were what we prepared for our User Testing 2.
Feel free to view the schedule and User Testing execution process that has been prepared, as shown in the first external link. The second link is the feedback form that all testers will fill in after trying out the testing.
Prepared Documents | Link |
---|---|
User Test 2 Schedule, Execution Process | UT 2 Schedule and Execution process |
User Test 2 Feedback Questions | SMU Quadtrics Feedback Questions |
Download UT 2 Feedback | |
User Test 2 Plan | Download UT 2 Plan |
User Test 2 Video Tutorial Transformation Process | Our Video Tutorial Improvements and Timeline |
User Test 2 Instructions for Testers | Download instructions |
UAT Results/Analysis | UT 2 Feedback Form Results |
User Test 2 Schedule and Final Results | UT 2 Observation Results |
UT 2 Setup
Introduction
After our MidTerm presentation, we decided to improve our User Testing process, especially since we are creating an application on the Kinect platform, which is still a new technology in the industry as compared to mobile phones. Thus, we decided to implement black box testing, which will enable us to get both qualitative and quantitative data from this User Test 2. This will enable us to understand what are the main issues faced by the testers when placed in a quiet environment where they will not be able to ask the developers any questions pertaining to the list of tasks they will have to complete.
UT2 Goals
Collect useful usability and gesture feedback from our testers to better understand their needs and improve our existing application.
UT2 Objectives
- Unguided Test for all testers
- Verify if the user finds it intuitive and easy to pick up hand gestures
- Verify the usability and easy navigation of the application
- Gather user opinion on Kinect Application feasibility in a retail setting
Room setup
Equipment Used
Observers’ Tasks
Observer's points to note for each Tester | Observer's procedures |
Here are a list of points that we have to do to ensure that we captured all the relevant data from the tester.
|
Collocate log files for each participant/scenario Summarize each scenario across all participants, includes plots and trend analysis
Make conclusions for each test goal |
Difference from UT 1
Video Tutorial Improvements and Timeline
Please go to our Video Tutorial Improvements and Timeline which will show you 6 different versions over the past few weeks.
UT 2 Process
Black Box Testing
To ensure that our 2T1 meets our objectives in allowing the tester to use the application without our help, we made it a non guided User Test. We wanted to verify if the users could pick up the gestures naturally and provide us opinion on how our group can further improve our application. Thus, we will place instructions at the side of the tester (both left and right) and the user will follow the steps required to allow us to know which features require more help from the user, without the need of someone to provide immediate help.
Please go to our Video Tutorial Improvements and Timeline which will show you 6 different versions over the past few weeks.
UT 2 Results
Testers Demographics
Type | Number | Percentage |
---|---|---|
SIS Students | 25 | 71% |
Non-IS Students | 9 | 26% |
Professors and Instructors | 1 | 3% |
Comments: Non-SIS students represent the user population from other faulties in SMU and their comments tend to be much less technical, reflecting most of the actual target public users. In addition, they also require much more guidance than SIS students, who are more experienced in application development and new technologies. Non SIS students provided us with an interesting dimension to the feedback collected, such as more of the business feasibility of the Kinect application, as compared to the SIS Students who will be more inclined towards the technology, and will tell more of which functions are more useful.
The instructor, on the other hand, is also very technical and provided us the best advice and opinions for a user to pick up gestures naturally, and how the application will be suitable for the public, based onher vast experiences with lots of applications.
In addition, we had a good mix of males and females, giving us a good mix of reviews, not bias only to one gender.
Type | Number | Percentage |
---|---|---|
Male | 19 | 54% |
Female | 16 | 46% |
User Testing 2 Outcome
Apart from quantitative results, our team has received very positive feedbacks and suggestions on how we can improve our application. Most of the testers feel that the Kinect application is cool, interactive, and still room to improve in the Augmented Reality function and the gestures used in the application. They see a potential in this application but a minority of 3 testers feel that they do not want to use the application again. All other testers will want to use our application again.
User Testing 2 Documents
For more information on our UAT 1, you can download our documents below:
Documents | Link |
---|---|
UAT Results/Analysis | UT 2 Feedback Form Results |
User Test 2 Schedule and Final Results | UT 2 Observation Results |