IS480 Team wiki: 2012T1 M.O.O.T/Final Wiki
- 1 Project Progress Summary
- 2 Project Management
- 3 Quality of Product
- 4 Reflection
Project Progress Summary
Download our Final Presentation Slides for your reference during our Final Presentation(0930, Monday 3 December).
- Inability to implement unsupervised learning
- Percentage of men picking male hat and vice versa during UT2 was only 27% - not reliable
- Cumbersome preparation for User Testing at Plaza Singapura
- Tricky gesture choices for interactive flow requiring numerous times of gesture changes
There are also other midterm project highlights.
Balancing Intuitiveness and Technical Competence
Natural User Interface poses a challenge as something intuitive for us is often not easily translated into something that machine can understand. For example, it is natural for us to point at something to indicate that we have chosen the thing, in our case, a door. However, Kinect does not detect finger, hence we need to think of other ways to indicate choice that is not awkward, yet still implementable. We usually come up with a list of possible gestures and intentions, and eliminate them systematically. The same applies for other gestures implemented in AlterSense.
Securing User Testing Site at Plaza Singapura
As CapitaMalls Asia is a large Trans National Company, each employee's job scope is very specific, hence there is a myriad of people we have to liaise with just to secure the testing site. It was difficult to communicate with so many parties, and the assumption that messages would be passed on to relevant people often did not materialize. We received a long list of requirements for "renting" a site a week before the scheduled User Testing (UT). Since none of us had any experience of renting commercial space for roadshow, we struggled in understanding and attempting to meet the requirements, as well as in coming up with contingency plans. Fortunately, through tighter communication with relevant key people and a lot of persistence, we managed to hold the UT as scheduled.
- Tested by real shoppers of diverse demographics
- Possibility to be further developed in phases by CapitaMalls Asia
- More than 80% gender prediction accuracy in controlled environment
- More than 70% gender prediction accuracy in uncontrolled environment
- Developed a readily deploy-able motion sensing gender predicting system from scratch
The two aforementioned documentations also aid us in establishing actual vs planned schedule, which comes in handy in monitoring outstanding tasks and to inspect whether the delays inflict much impact on the iterations that they follow.
Overall, we were able to follow our planned schedule quite closely. Major challenges were encountered in Iteration 1, as described in our midterm project summary. Post midterm, the most challenging period resulting in relatively significant delay was in Iteration 6. Preparation for User Testing at Plaza Singapura took development time away from Russell Goh Zhi Wei,who had to liaise with electrical contractor to pull power out of the insulator and set up cabling setup that fulfills required regulation. As a result, all Advertisement Management System-related features and other tasks assigned to Russell were delayed.
Schedule & Bug Metrics
We keep track of schedule and bug metrics, and the metrics have been useful external cues in directing the project.
Overall, velocity of 17 is our average, stable velocity - indicating that in an iteration, team can be reliably expected to complete tasks of worth 17 points. The points were assigned based on perceived complexity or time required to complete the tasks. There was a schedule slip in Iteration 6 due to the unexpected amount of extra effort required in securing a site for User Testing in Plaza Singapura. Time was hence channeled from completing development task to settling the requirements for User Testing. Nonetheless, team caught up and recovered to the stable velocity of 17 in the following iteration.
From iteration 5 onwards, we met several bugs that caused the total bug score to be above 10, raising the red flag that the team needed to spend more time debugging before proceeding. Fortunately, the team was able to solve all the bugs within the respective iteration, hence there was no need to push the solving of bugs to the next iteration nor was there a need to drop any functionality.
Gender Recognition Metrics
Our gender recognition metrics can be accessed here.
We used leave-one-out cross-validation to find the optimum training set size that yields the highest gender prediction accuracy. Several rounds of testing shows that the minimum amount of training sets needed should be at least 60. More data is required to achieve an accuracy above 90%.
A round of testing of the integration of the hair and skirt parameters as biased inputs into the Neural Network has shown that the hair and skirt parameters were affecting the accuracy of gender prediction due to their relatively low accuracy of determining the presence of hair and skirt respectively. As a biased input has a much higher weightage than a non-biased parameter in influencing the final outcome of gender prediction, the final accuracy of gender prediction has thus dropped significantly.
Since gender prediction works fine with just the three non-biased parameters (height, shoulder & hip width) as shown during our UT1 - with the accuracy reaching 87%, we decided not to introduce any biased parameter to influence the final outcome of gender prediction.
Front End Development
The front end development of AlterSense involves developing a scene manager that ensures every scene can be developed without manipulating the properties of various graphical user elements in the previous or upcoming scenes. We had to implement Boolean variables and methods that could manipulate the properties of a graphical element. For instance, the Windows Presentation Foundation layer framework offers a storyboard class that enables animations such as the fade effect of the Speech Bubble in AlterSense. We also had to develop other methods that would control the visibility, position, size and other properties to aid front-end development.
Front-end development requires a lot of debugging and testing due to the nature of timing, sequence and aesthetics. Considerations such as whether the object appeared at the right moment, is large enough for users to see, yet it should not obstruct other elements in the process is a balancing act altogether. There has to be careful sequencing when it comes to invoking various methods, and ultimately, the overall result should not promote a bad user experience.
Preventing Object from Going Out of Frame
With regards to user experience, we encountered an issue that graphical objects would go out of the sight if the user moves to the edges of the Kinect Camera view. We identified this to be a bad user design, and created methods to ensure that AlterSense would know when a graphical object is out of frame, and re-adjust the parameters accordingly so that it will always remain on the screen.
To do this, we have to dynamically identify the height and width of the object as well as retrieve the boundaries of the canvas. Next, we identified the 4 corners of an object and monitor the positions of these 4 points in real-time. When the shopper moves to the edge of the camera view, one or more of the object’s positions will collide with the edges of the canvas object, and AlterSense would stop adjusting the positions of the object, and set the positions such that it would be viewable. When the shopper moves back into an acceptable boundary, AlterSense would ensure that the graphical object would continue to move within the acceptable boundaries. The overall effect of this implementation ensures that instructions and images will always be viewable within the screen, regardless of how the shopper positions himself in front of the Kinect.
The entire project involves several systems that are interconnected. Deciding which method to link these systems requires some consideration such as the speed, complexity and scalability. During the development of Ad Management System (AMS), the mode of accessing the tag by AlterSense (in providing the content required for targeted advertising) via online or offline method needs the following clarification:
- How often do the client update the ads?
- Are the ads time sensative?
- What’s the infrastructure available at the deployment area?
After weighing and exploring several options, File Transfer Protocol (FTP) and batch update via SQL stored in database was considered. Eventually batch updating via SQL was choosen due the ease of managing data in a database. Additional information could alsobe stored in the database to enable search capabilities and due to the scalable capability of database, storing tags in binary seems to be the most ideal option for AMS.
Enabling Dynamic Connections
The Microsoft Tag SOAP web service is an external service provide by Microsoft which AMS serves as an interface to manage all ads and data associated with each tags. The AMS is required to be designed to be enable dynamic change of end-point connected to the account associated. The current token used by AMS is only allow for academic purposes, whenever CMA choses to purchase the license to create tags, the associated token could be easily changed.
Quality of Product
|Project Management||Metrics||Schedule & Bug Metrics, Bug Log|
|Minutes||Client Meeting,Supervisor Meeting|
|Content||Interaction flow chart||Flow Chart|
|Narrative flow||Interactive Content Flow|
|Design||Use Case||Use Case|
|Architectural diagram||Architectural Diagram|
|Testing||User Testing 2 documentation||UT2, UT2 Supporting Documents|
- AlterSense is a stand alone application that can be deployed to many shopping malls in Singapore
- Storing of gender-targeted tags in Advertisement Management System (AMS, which is easily accessible by client allows client to add more content
- Gender-targeted content is retrieved from AMS, which offers an easy interface to manage (create, update, delete) content online
- Scenes and gestures in AlterSense storyboard are decoupled, allowing amendment of scene sequence or interaction to be implemented without inflicting drastic changes to other scenes, as mentioned in Front End Development technical complexity
Using hexadecimal name instead of color.x method (where x is the name of the color) to color pixel for background and outline improved performance as it is a more direct method, resolving the issue of lags we encountered in the initial stage
- The use of Neural Network facilitates machine learning - client can attempt to further improve the accuracy of gender recognition by acquiring more training data, although our data analysis has shown that more than 60 training data is able to produce more than 70% accuracy
- MySQL credentials and Microsoft Tag token is stored in separate files to enable easy configuration
- A switch has been created for Supervised Learning scene - client can choose to configure (activate it or replace the scene with a more reliable one) it accordingly
- Numerous front end development features such as dynamic callout bubble, preventing interaction object from going out of frame,as explained in Technical Complexity allows real shoppers to use AlterSense with considerable ease - as observed during User Testing 2
- Automatic refreshing of scene, error controls, and instructions allow shoppers to interact with AlterSense even without guidance
Code is organized into controller classes based on functions, allowing straight forward editing and updating of code
Decoupling AMS from the front end-gender recognition parts of AlterSense improves fault-tolerance. Even if the Microsoft Tag SOAP service is down, downloading of tag will still be available, just that creation and editing of new tags will be affected
We handed over a readily deploy-able system to CapitaMalls Asia (CMA). The deliverable delivered for User Acceptance are:
- Code executable
- Pre-requisites software and framework
- Deployment manual
- Pitch video to show an overview of scenes and story board
- Project documentation - content from Wiki in pdf format, as requested by client
Hardware used for development are CMA properties and will be returned accordingly, and can hence be used for actual deployment as well.
|Date||5 November 2012 from 12pm to 6pm|
53 testers - 29 females & 24 males
1. Ask shopper if they have a smartphone and to download Microsoft Tag if they have one. If they are unwilling to download, we lend them our phones
"Team M.O.O.T has shown remarkable tenacity and creativity in coming up with brilliant ideas to help the client
and have performed to the best of their abilities to make things happen."
Manager (Information Technology)
CapitaMalls Asia Limited
We simply cannot believe that we have made it this far. The very initial idea of coming up with an interactive mall directory was turned down by CMA at the blink of an eye, rendering weeks of brainstorming and preparation to pitch the idea in vain. Nonetheless, if Edwin (IT head of CMA) did not urge us to look for something with a WOW factor, something novel, something that would excite shoppers; we would not have come up with AlterSense - and might indeed have ended up with a project that we ourselves might not be proud of.
AlterSense is a dream comes true for the team - we were inspired as we watched a video of wall display displaying intriguing visualizations as a person walked past the wall. However, there were a myriad of challenges in materializing our desired interactive wall display: none of us has any background in visual design, what was the business value, what kind of story board could be crafted out of it; not to mention the challenge of picking up C# and venturing into hardware calibration as well.
The success of the team in completing the project would not have been possible without the support of two people: our supervisor, Jason, and our main point of contact with CMA, Ethan. Jason has assisted the team to great extent in trimming down the scope and in crafting a holistic business solution. Jason encouraged the team to go back to the drawing board often to come up with a more cohesive story board - it might seem tedious, but the results were apparent. Without so doing, AlterSense could have ended up as a pool of disparate technological exploration, instead of a complete solution with interactive narrative flow that it is now. Ethan has been a great support to the team - our invaluable mediator who was willing to stay back after office hour to listen to our ideas and updates, who tried to voice our concerns to the other parties concerned in CMA, who readily assisted us with logistics procurement, and supported us from the project inception to its completion. Definitely, along the course of project, we have received help and encouragement from other sources too. Professors whom we consulted on Neural Network and data mining, Professor Ben who linked us up with CMA after finding out about our mall directory idea, as well as other faculty members and CMA employees who have assisted us in one way and another.
Generally, we are satisfied with how far AlterSense has come to. Nonetheless, if we could do things differently, we would like to try using decision tree in determining gender. Doing so will give us more control and insights into how gender is determined by AlterSense. Although Neural Network (NN) was chosen due to the fuzziness of the classification required as well as its ability to do machine learning, NN is a black box - nobody knows how exactly the inputs are processed to determine gender (output). Although it still works to a level of decent accuracy and is indeed suitable for our client's needs, knowing how exactly it works and having more direct control over the prediction process may allow further expansion of the gender recognition feature.
We have picked up new technical skills from this project, no doubt. Nevertheless, most importantly, we have learnt to be resilient and persistent - in pitching idea, in defining our scope, in materializing something that we would like to see. We perceive it as a life skill - we would certainly face difficulties in our future endeavor, may we then be able to look at Project AlterSense on hindsight and recall that we can achieve our goals if we persevere: learn if we do not know how to, seek if we do not have, ask if we are not given.
This journey has been a challenging, yet fulfilling experience for me. As our project focuses on shopper's experience, it is essential to always keep the shopper in mind while designing the storyboard. Whenever there was a change in requirements or we needed to add something new into the storyboard, we had to take into consideration many factors such as whether the addition would be too abrupt and destroy the flow of the storyboard or whether the experience would be intuitive enough without being too straight in the face. Ensuring that all the various sections are integrated together with a consistent storyline and common theme while keeping the fun factor was definitely a very challenging task. In addition, from our User Testing 2, I realized that the perspective of being a tester of an application is very different from being the creator of the application. Certain things that we thought would be natural and intuitive to a person did not turn out to be so. Hence there was a need to change our perspective and start thinking of the shopper as one that would requires a lot of hints.
Through this project, I have also picked up valuable soft skills such as working with people with different working styles. Having a diversified team with each team member having different strengths certainly helped us in leveraging on each other's strengths. In addition, learning to understand one another's weaknesses and think of ways to adapt and work around it instead of expecting others to change definitely was one of the learning point.
Personally, I am very grateful to be part of this exciting project with this special bunch of people and am very glad that our team have managed to come this far. It is definitely a memorable experience which would help guide me through my future.
Project AlterSense trained me not only to be well-versed in my development skills, but also soft skills such as logistics and people management. It was evident in our last user testing in Plaza Singapura, where we went through several hurdles such as coordinating with several parties to approve the location for testing (people from the Mall, IT and MarCom Department), to even liaising with technician to prepare power points for our test location. These incidents trained me in areas that couldn’t be taught in an academic setting, equipping me with skills that would be essential in real world setting.
When developing AlterSense’s Kinect Gesture, the thought process behind each gesture was vital in making the user experience enjoyable and fun. Using Kinect challenged us to design and create visual cues for users, to understand what kind of gesture is required for the desired input. Something as simple as choosing a door could be difficult to be understood by a machine. We had to brainstorm, what would be the best method for a user to select a door? Can the user complete an action without any instruction given? Designing a natural user interface for user to interact with was the most challenging part for me. Through our user testing and feedback, I was able to gain more experience and insight in this field and able to put myself in the shoppers’ perspective when developing.
Overall my IS480 journey had been fulfilling, working with CapitaMalls Asia allows me to pick up business acumen that would be valuable for my career in the future. Having an awesome team and friends to share this journey with me, IS480 would definitely be the most memorable project in SMU.
I used to think that schedule planning was more of a perfunctory administrative matter. This project, however, shows me that the project schedule is,in fact, the thing that keeps everyone on the same page. It lets us know where we are, what else are left to do. Iterative methodology allows me to revisit the schedule at the end of iteration, and craft the schedule for upcoming iterations based on our progress in the past iteration. Pivotal Tracker has been a handy tool in determining velocity, which I leverage on as a gauge of how much we can accomplish in iteration.
Many times we were questioned on the feasibility of using Kinect in shopping center, or of using Neural Network instead of regression to determine gender; just because the aforementioned things are not common and put us in a state with very steep learning curve. Equipped with the willingness to learn and an open mind, voila! We have actually come so far! This project instills this realization that I can learn new things and apply them to the context that I want to see them in.
Lastly, I witnessed the pivotal role of business-IT alignment in this project. Balancing narrative flow that provides shopper with satisfying user experience and reasonable features development is an art. We may be tempted to include a feature - just because it is technically reasonable, but have to eventually go back to the drawing board to find alternative way because the initial proposed feature simply does not fit the story line.
All in all, Project AlterSense has provided me with a a glimpse of end-to-end, real world software development. The skills obtained definitely make me feel better equipped to take up other challenging projects in the future.
The IS480 journey has taken my learning beyond the classroom. Meeting a real client and developing a Kinect project was an exciting and rare opportunity to speak with external parties and obtain a realistic perspective to project management. From a project perspective, it is important to be flexible as changes are bound to happen. This project has gone through several rounds of requirement changes over time, and I believe it’s important to keep the development modules independent so new changes can be implemented more readily. Additionally, I have learnt a lot about developing business requirements as my client offered business domain knowledge as well as meeting other stakeholders from various departments within CapitaMall Asia. Meeting these stakeholders offered valuable insights, and understanding their concerns is a very essential element required to develop the business requirements of AlterSense.
In hard skills, I have gained knowledge in Kinect development, Windows Presentation Foundation layer and various methods to process data streams. In addition, I am more familiar with front-end development as controlling the user interface elements and adjusting them to fit the storyboard proved to be a delicate affair that required vast amounts of testing and careful sequencing of methods. On top of getting things to work, a front-end developer has to place more emphasis in aesthetics, sequence and overall layout.
AlterSense is definitely one of most challenging projects that I will remember. Coming to the end of this project, I am personally satisfied with the outcome. Having everything pieced together to form a working solution that fits CMA’s goals is a fulfilling task. I gained a lot from this project, and believe that these experiences would be useful in my future endeavors.
|I feel grateful for the opportunity to experience this IS480 journey with my fellow team members. At a project level, I learnt a lot about managing the expectations of multiple stakeholders. I felt that it was a rewarding experience because it pushed the boundaries of what our team could achieve, given the time constraints and technological complexities. I also learnt how to appreciate the process of exploring into unknown territory, without having any prior knowledge about a subject matter. At a personal level, I learnt a lot about risk management through juggling multiple projects and prioritising to-do items. Moving forward, I hope to pass on this experience to fellow SMU SIS students who will be embarking on their IS480 journey in the future.|