IS480 Team wiki: 2012T1 M.O.O.T/Technical Applications
User Testing 1 | User Testing 2 | Development Technologies | Hardware | Primary Research | Secondary Research |
Contents
User Testing 1
Our UT 1 took place on 24 September 2012 from 12pm to 4.30pm.
Objectives
- Test the accuracy of gender recognition in a real live setting
- Gather users’ opinions about photo taking using augmented reality
Setup
Location: The T-junction at the school’s basement is selected because it
- Emulates a real live mall setting
- Has a comparatively high human traffic so as to garner as much feedback as possible
- Has 7m * 6m (~452 sq ft) of floor space
- Has a plain wall background on one side of the booth (as Kinect sensor performs best with plain background)
- Has 1 power outlet to provide power for laptop and equipment
The setup at the T-junction will follow as stated here while the overview of setup layout will be as shown:
Scope completed
- Gender Recognition through:
1. Detection of height
2. Detection of shoulder width & center of moment
3. Detection of presence of skirt
4. Detection of presence of hair
5. Detection of presence of bag
However only the 1st two parameters (height, shoulder width) were integrated into Neural Network - AForge learning algorithm
- Prototype of photo taking
1. Countdown timer (to signal the start of photo taking)
2. Snapping of photo
Testers
We have a total of 70 testers – 31 females and 39 males. As one of our focus is to check the accuracy of our gender recognition algorithm, we picked our testers in the following way:
Female | Male | |
Height | 14 with height ≤ 1.6m 14 with height > 1.6m but < 1.7m 3 with height ≥ 1.7m |
13 with height ≤ 1.7m 19 with height > 1.7m, but < 1.8m 7 with height ≥ 1.8m |
Skirt | 19 not wearing skirt 12 wearing skirt |
N.A |
Hair | 10 with short hair (does not touch the shoulders) 21 with shoulder-length or longer hair |
N.A |
Bag | 1 carrying bag on their elbow 30 not carrying bag on their elbow |
N.A |
Center of moment (COM) | We found 2 pairs of females and males that have similar height and similar body proportions |
Standard of Procedure (SOP)
1. Let tester use AlterSense by himself/herself
2. Record down the details of each tester into [Gender Recognition Metrics]
3. Record a short video of tester using Kinect Studio
4. Let tester fill up a survey form
Results of survey form
Feedback on gender recognition
Based on these results, we decided to have neutral promotions that are suitable for both gender groups for shoppers that we don’t have full confidence in detecting their gender since 37% of the people surveyed have indicated that they would be offended if they were recognized of the wrong gender.
As 58% of the people surveyed indicated that they are more likely to purchase an item from a targeted promotion that is specific to their gender group as opposed to that from a neutral promotion suitable for both gender groups, this justifies our main objective of having targeted advertising to enhance shopper’s experience.
Feedback on photo taking
All of our testers had no difficulty taking a photo with the majority finding it very easy to take a photo with AlterSense.
Around 40% of the testers found that the countdown timer which was used to signal the start of photo taking, was confusing and not intuitive at all. Among them, some did not notice the presence of the countdown timer. Thus we have decided to implement a speech bubble with words such as “Let’s take a photo” to make the signal of starting photo taking more intuitive.
Results of accuracy of gender recognition algorithm
Overall gender accuracy
The gender recognition is pretty accurate at a rate of 87.14% after testing 70 people. Some of the reasons of the inaccurate detection of gender were due to the inability of Kinect to handle reflective items such as polished shoes or shiny pants. This caused the legs of the tester to be cut short as Kinect was not able to detect the full length of the legs.
To counter this, we have decided to have input validation to check for heights that are ridiculously low and check whether the image was chipped off or not. When the height detected is 1.4m and below and the image is found to be chipped, we will deem the input as invalid and show neutral promotions to the shopper instead.
Skirt
The method of detecting skirt is fairly accurate at a rate of 62.86% with 42 correct detections out of 70 detections. We are planning to refine this method by calculating whether there is a gap between the legs. Since most people tend to stand with their legs slightly apart, females who are wearing skirt would tend to not have a gap between their legs as compared to males who are wearing shorts or pants and females who are wearing shorts or pants.
Hair
The method of detecting length of hair is not very accurate at a rate of 38.57% with 27 correct detections out of 70 detections. We are planning to refine this method by calculating the width of the head and comparing it against the width of the neck. As most females with their long hair let down tend to have a silhouette neck width that is similar to the width of the head, we would determine the presence of long hair by this.
Bag
The method of detecting bag on elbow is not very accurate at a rate of 22.86% with 16 correct detections out of 70 detections. This is due to the fact that the testers who are carrying a bag on their shoulder would have their arm coordinates messed up as Kinect would detect the bag as part of the tester's arm. Also, many people are carrying or holding onto items that are not gender-specific (i.e. items that only females would carry). Such items include bubble tea, food items or plastic bag. As Kinect is unable to differentiate the type of item a person is carrying, it would detect any item as a bag. Because of the above two reasons, we have decided to drop this method of detecting bag.