IS480 Team wiki: 2012T1 M.O.O.T/Technical Applications
|User Testing 1||User Testing 2||Development Technologies||Hardware||Primary Research||Secondary Research|
- 1 User Testing 1
- 2 Objectives
- 3 Setup
- 4 Scope completed
- 5 Testers
- 6 Standard of Procedure (SOP)
- 7 Results of survey form
- 8 Results of accuracy of gender recognition algorithm
- 9 Photos of UT1
- 10 Photos From Photo Taking
User Testing 1
Our UT 1 took place on 24 September 2012 from 12pm to 4.30pm.
- Test the accuracy of gender recognition in a real live setting
- Gather users’ opinions about photo taking using augmented reality
Location: The T-junction at the school’s basement is selected because it
- Emulates a real live mall setting
- Has a comparatively high human traffic so as to garner as much feedback as possible
- Has 7m * 6m (~452 sq ft) of floor space
- Has a plain wall background on one side of the booth (as Kinect sensor performs best with plain background)
- Has 1 power outlet to provide power for laptop and equipment
The setup at the T-junction will follow as stated here while the overview of setup layout will be as shown:
- Gender Recognition through:
1. Detection of height
2. Detection of shoulder width & center of moment
- Behavioural parameters of gender recognition:
1. Detection of presence of skirt
2. Detection of presence of hair
3. Detection of presence of bag
- AForge learning algorithm
- Prototype of photo taking
1. Countdown timer (to signal the start of photo taking)
2. Snapping of photo
We have a total of 70 testers – 31 females and 39 males. As one of our focus is to check the accuracy of our gender recognition algorithm, we picked our testers in the following way:
|Height||14 with height ≤ 1.6m
14 with height > 1.6m but < 1.7m
3 with height ≥ 1.7m
|12 with height < 1.7m |
20 with height ≥ 1.7m, but < 1.8m
7 with height ≥ 1.8m
|Skirt||19 not wearing skirt
12 wearing skirt
|Hair||10 with short hair (does not touch the shoulders)
21 with shoulder-length or longer hair
|Bag||1 carrying bag on their elbow
30 not carrying bag on their elbow
|Center of moment (COM)||We found 2 pairs of females and males that have similar height and similar body proportions|
Standard of Procedure (SOP)
1. Get tester to stand at a specific distance away from the Kinect
2. Let tester use AlterSense by himself/herself
3. Record down the details of each tester into [Gender Recognition Metrics]
4. Record a short video of tester using Kinect Studio
5. Let tester fill up a survey form
Results of survey form
Feedback on gender recognition
Based on these results, we decided to have neutral promotions that are suitable for both gender groups for shoppers that we don’t have full confidence in detecting their gender since 36% of the people surveyed have indicated that they would be offended if they were recognized of the wrong gender.
As 73% of the people surveyed indicated that they are more likely to purchase an item from a targeted promotion that is specific to their gender group as opposed to that from a neutral promotion suitable for both gender groups, this justifies our main objective of having targeted advertising to enhance shopper’s experience.
Feedback on photo taking
All of our testers had no difficulty taking a photo with the majority finding it very easy to take a photo with AlterSense.
Around 40% of the testers found that the countdown timer which was used to signal the start of photo taking, was confusing and not intuitive at all. Among them, some did not notice the presence of the countdown timer. Thus we have decided to implement a speech bubble with words such as “Let’s take a photo” to make the signal of starting photo taking more intuitive.
Results of accuracy of gender recognition algorithm
Overall gender accuracy
The gender recognition is pretty accurate at a rate of 87.14% after testing 70 people. Some of the reasons of the inaccurate detection of gender were due to the inability of Kinect to handle reflective items such as polished shoes or shiny pants. This caused the legs of the tester to be cut short as Kinect was not able to detect the full length of the legs.
To counter this, we have decided to have input validation on Kinect to check for heights that are ridiculously low and check whether the joints are clipped not. When the height detected is 1.4m and below and the joints are found to be clipped, we will deem the input as invalid and show neutral promotions to the shopper instead.
Note that the gender recognition method is currently accurate only when the tester is standing at a specific distance from the Kinect. But we will continue to refine the method to ensure that the gender recognition is accurate regardless of where the shopper is standing from the Kinect.
* To Align
* To Align
There is room for improvement for gender recognition of taller females and vertically challenged males. To allow our Neural Network to better distinguish the differences between taller females and vertically challenged males, we plan to get more data sets of measurements by conducting 2 sessions of measurements taking in Week 8 and 9. With a larger data set to feed the Neural Network, the learning process of the Neural Network will be enhanced, leading to an improvement in accuracy.
* To Align
The method of detecting skirt is fairly accurate at a rate of 62.86% with 42 correct detections out of 70 detections. We are planning to refine this method by calculating whether there is a gap between the legs. Since most people tend to stand with their legs slightly apart, females who are wearing skirt would tend to not have a gap between their legs as compared to males who are wearing shorts or pants and females who are wearing shorts or pants.
* To Align
The method of detecting length of hair is not very accurate at a rate of 38.57% with 27 correct detections out of 70 detections. We are planning to refine this method by calculating the width of the head and comparing it against the width of the neck. As most females with their long hair let down tend to have a silhouette neck width that is similar to the width of the head, we would determine the presence of long hair by this.
* To Align
The method of detecting bag on elbow is not very accurate at a rate of 22.86% with 16 correct detections out of 70 detections. This is due to the fact that the testers who are carrying a bag on their shoulder would have their arm coordinates messed up as Kinect would detect the bag as part of the tester's arm. Also, many people are carrying or holding onto items that are not gender-specific (i.e. items that only females would carry). Such items include bubble tea, food items or plastic bag. As Kinect is unable to differentiate the type of item a person is carrying, it would detect any item as a bag. Because of the above two reasons, we have decided to drop this method of detecting bag.
Photos of UT1
|Our setup||In the midst of photo taking|
|Inez guiding our tester||Our testers filling up the survey form|
|Many many people||Our kind testers who gave us a lot of valuable feedback|