IS480 Team wiki: 2012T1 Team Sageby Midterm Wiki
||||PRODUCT OVERVIEW|||||PROJECT MANAGEMENT|||||USER TEST|||||LEARNING OUTCOMES|
- 1 Project Progress Summary
- 2 Project Management
- 2.1 Project Timeline
- 2.2 Schedule Brief
- 2.3 Project Status
- 2.4 Project Schedule (Plan VS Actual)
- 2.5 Project Metrics
- 2.6 Project Risks
- 2.7 Technical Complexity
- 3 Quality of Product
- 3.1 Intermediate Deliverables
- 3.2 Deployment
- 3.3 Testing
- 3.3.1 Unit Testing
- 3.3.2 Test Cases
- 3.3.3 User Test
- 188.8.131.52 UT 3.1
- 184.108.40.206 UT 3.2
- 4 Reflection
Project Progress Summary
Since started off as a passionate team of 5 in May, we have gone through 4 iterations of construction, learning and verifications. We have strictly followed our core methodology, to Build, Measure and Learn that is adopted from Lean Startup. This methodology not only teaches us to change/pivot when necessary, but also to prevent wastage of efforts and time. Not only do we have to grab every opportunity to verify assumptions, build with leaner resources and learn from results, we have to understand client's needs and handle ever-changing situations.
What we have done thus far:
- Built the application that can be used to do register with Sageby and create an account, do a survey and to share or feedback to Sageby Surveys
- Constructed the Application Programming Interface that provides support for User and Survey data retrieval
- Executed Beta Testing and Live Testing with more than 100 users combined
- Carried a real survey by LTA that provides real rewards
- Transitioned from Development Database to Live Database
- Transitioned from iOS5 to iOS6
- Deployed twice in Appstore and garnering 80 downloads till date
Despite having a clear schedule and backed with good responses from Clients, there are still unseen situations that arises and needed our attention to either rectify the problem or grab the opportunity that improves our product.
|1||Construction of Application Programming Interface (API)|
To receive dynamic data and move beyond a Proof-Of-Concept product, Sageby graciously agreed to help to build the APIs that are necessary. However, their lead developer unforeseeably had to leave Sageby due to personal reasons.We had to make a drastic addition to our schedule by squeezing in time and manpower for building API while removing non-functional aspects such as testing for Performance Profiling and Memory Leaks.
Through this initiative, we managed to build the necessary APIs in time for our Live Test / User Test 3 that is an extremely important milestone in our project.
|2||Transition from Development Database to Live Database|
Sageby set out numerous stringent tests that we had to comply with before allowing us to access their live database. Thankfully, our diligence paid off and Sageby felt confident with having the application to access the live server.
However, before that, we had to clear several legacy issues such as MySQL plugin versioning issues, different PHP versions and unfortunately, face a database account user lockdown.
After ironing out issues and successful uploading of APIs, we managed to move from dummy data to real users data and this brought us to the next level in product development, from a Proof-Of-Concept to a usable and productive Application!
|3||Live Testing with LTA Survey & Real Rewards|
After carrying out the transitioning of database and inserting of API queries and codes, we managed to finish our iteration in time for User Test 3. However, Sageby proposed another idea that allows us to carry a real survey by LTA and to give out real rewards. Considering the fact that User Test 3 is targeted at a wider pool of users, we had to make changes to the current UT plan and accomodate the LTA survey.
This inclusion of LTA survey really showed us the true potential of the application as it shows that not only users are fine with doing surveys on mobile, many agreed that a niche type of survey can be dedicated for the mobile platform, Short Survey Polls.
|Task/function/features, etc||Status||Confident Level||Comment|
|Login Module||Fully deployed and tested 100%. However, it is not updated to latest Facebook SDK due to lack of backward compatibility with iOS 5||1||Luojia|
|Survey Module||Fully deployed and tested 100%, with categorization via alphabetical order||1||Luojia|
|Rewards Module (Store)||Fully deployed and tested 100%, with categorization via alphabetical order||1||Luojia|
|Rewards Module (Wallet)||Fully deployed and tested 100%, with categorization via used & unused sections||1||Luojia|
|Settings Tab||Fully deployed and tested 100%||1||JunHao & Mervyn|
|Social Sharing Module||Fully deployed and tested 100%||1||TeeJun|
|Feedback Channel||Fully deployed and tested 100%. Used Apptentive Package for feedbacks and easy rating portal.||1||Junhao|
|Performance Profiling & Memory Leak Tests||Function to be Kept in View. To be continued if necessary.||1||Marcus|
|Performance Profiling & Memory Leak Tests||Function to be Kept in View. To be continued if necessary.||0.5||Marcus|
|Application Programming Interface||Partially developed, deployed and tested 50%. Survey, Users and Rewards Module completed||1||Marcus|
|Database Transition from Development to Live||Fully deployed and tested 100%||0.8||Marcus|
|Unit Tests||Partially developed, deployed and tested 50%. Login, Surveys, Rewards, User Module & Settings tab covered||0.8||JunHao|
|Facebook Sharing UIView||Partially developed, deployed and tested 50%.||1||Mervyn|
|Mobile Template for Lime Survey||Fully deployed and tested 100%||1||Mervyn & Junhao|
|Social Integration||Partially developed, deployed and tested 30%. Twitter integration and Facebook Login||0.8||Luojia|
Project Schedule (Plan VS Actual)
Despite having several additions in our project scope, we managed to narrow down the non-functional requirements of the applications and added an extra week in Iteration 4 to accomodate our User Tests.
|1||Complete Paper Prototype for client's acceptance||30/5/12||5/6/12||30/5/12||5/6/12|
|Develop sample iOS App||6/6/12||13/6/12||6/6/12||13/6/12|
|Complete Prototype with Survey Module||14/6/12||27/6/12||14/6/12||27/6/12|
|2||Build Facebook Single Sign On Auth||27/6/12||3/7/12||27/6/12||3/7/12|
|Build Sageby Manual login||3/7/12||11/7/12||3/7/12||11/7/12|
|Allow user to logout||3/7/12||11/7/12||3/7/12||11/7/12|
|Build User details popup||11/7/12||17/7/12||11/7/12||17/7/12|
|Build different in-house survey layouts. Allow survey question to have dynamic survey layout||17/7/12||25/7/12||17/7/12||25/7/12|
|Able to pull dummy data from Sageby's server||17/7/12||25/7/12||17/7/12||25/7/12|
|Build survey's question routing function||17/7/12||25/7/12||17/7/12||25/7/12|
|Deploy and upload to Apple's Appstore||31/7/12||31/7/12||31/7/12||3/8/12||Code-signing error and outdated library links led to 3 days delay in deployment|
|3||Build browsable table for Store vouchers||1/8/12||8/8/12||1/8/12||8/8/12|
|Allow users to redeem vouchers||1/8/12||8/8/12||1/8/12||8/8/12|
|Build browsable table for Wallet vouchers||8/8/12||15/8/12||8/8/12||15/8/12|
|Build QR generator||8/8/12||15/8/12||8/8/12||15/8/12|
|Deployment on Appstore||21/8/12||21/8/12||8/9/12||8/9/12||Pushed to Iteration 3 due to the lack of APIs|
|4||User is able to share completed surveys via Facebook share and Twitter||22/8/12||29/8/12||22/8/12||29/8/12|
|Users will be able to provide feedbacks to Sageby Surveys||22/8/12||29/8/12||22/8/12||29/8/12|
|User is able to share referral link for benefits||22/8/12||28/8/12||22/8/12||28/8/12|
|Build iFrame to support Lime survey||29/8/12||4/9/12||29/8/12||4/9/12||Added an Extra week to support external survey engine|
|Build iFrame to support Lime survey (Added)||-||-||29/8/12||4/9/12||Needed to support external survey engine for live testing(UT3)|
|Built User & Survey Module's APIs (Added)||-||-||5/9/12||11/9/12||Added an Extra week to build APIs to receive dynamic data and prepare for live testing (UT3)|
|Transitioned from iOS5 to iOS6 (Added)||-||-||12/9/12||18/9/12||Added an Extra week to transition from iOS5 to iOS6 in order to update deprecated APIs and to support application on iPhone 5|
|Transitioned from Development Database to Live Database (Added)||-||-||12/9/12||18/9/12||User will be able to register with Sageby and do real surveys.|
|Deploy to Appstore||25/9/12||25/9/12||30/9/12||30/9/12||Delayed by 5 days due to rejection from Appstore for several unsupported APIs|
Tasks with "-" planned dates are newly added tasks that was not initial in the project milestones.
We have chosen Schedule Metric and Bug Metric as it best suits a development for iOS due to expected rapid changes in schedule and unforeseen bugginess of a mobile app.
Issues Faced in Iteration 2
The team managed to complete new tasks on time in iteration 1. However, the technical complexity that we faced in iteration 2 was on a different level and we did not expect to face issues with customized survey styles and Facebook Login. This caused us to push our schedule back by 5 days.
Another issue was the delay of APIs from Sageby. This delay caused us to lose up to another 5 days in testing and development, and we had to hard code dummy data in order to meet the requirements for User test 2 for the iteration.
Issues Faced in Iteration 4
After Iteration 3, we had to re-scope iteration 4 due to several milestones to be met:
- Move beyond proof of concept
- Live Testing with real surveys and wide pool of users
- Allow users to register with Sageby
In iteration 2, we faced numerous issues regarding unknown durations for new tasks. In iteration 4, we revised through these tasks and scope out based on the team's experience with development. We had to add 2 weeks to iteration 4, out of the 3 buffer weeks intended, in order to accomodate 3 other major phases, iOS6, Live Database and construction of APIs.
The schedule metric has been highly useful for us by providing us with an better grasp of our current situation and to track the amount of time we took with difficult tasks. Project Manager was also able to efficient allocate extra time and also to meet up with changing demands and opportunities such as the unexpected chance of deploying a real survey for UT3.
We needed the Bug Metric to answer 3 important questions during development:
- Where did it originate from
- Impact of the bug
- How many are there
With the aid of the Bug Metric, we managed to identify key issues, which includes Facebook Login and sharing and persistent UI Bugs. Coding in Storyboard with Objective-C bypasses the conventional coding of transitions of Views in iOS. This is a huge improvement in convenience for new developers such as us, but when bugs arises, we need to identify the location of it and to rectify the issue by diving into the source codes and make manual changes.
Overall, the Bug Metric shows high correlation with the Schedule Metric and this is a relatively positive indication for the Project Manager that data collected responds well with the different Metrics in use.
We have split the risks into 2 components, Client and Project Management. Client's risk are rather intensive as Sageby is a new startup and they face plenty of risks that we have to take into account. Project Management on the other hand is rather standard but requires clear mitigation plan to avoid confusion within the team.
Client & Project Management
Technical Complexity takes into consideration of Team Sageby's main development pain points. Although there are other complexities such as Facebook login SDK issues and iOS 6 with backwards compatibility, these issues can be temporarily kept in view due to the immaturity of the SDK and platforms.
Quality of Product
Sageby Surveys formally launched on 18th of September and it comes with a full fledged working survey module which allows users registration with Sageby. The quality of product mentioned below focuses more on the non-functional aspects of the Application as functional tasks list are stated in the project scope. Also, these deliverables are constitutes the structure of the application and ensure its integrity and development modularity.
|Project Management||Meeting Minutes||Team meetings: Minutes 1-8|
|Supervisor: Minutes 1-3|
|Project Schedule Planning||Project Schedule: Project Schedule|
|Metric||Bug Metric: Bug Metric|
|Schedule Metric: Schedule Metric|
|Requirements||Midterm Presentation||Midterm Presentation Slides|
|Analysis||Use case||Use Case Diagram & Documentation|
|Functionality List||Functionality List|
|Design||Architecture System Diagram||System Architecture|
|Domain Diagram||Domain Diagram|
|Mockup Designs||Mockup Designs|
|User Test Documents||UT 1|
- Sageby Surveys Version 1.1 is currently deployed on Appstore
- Garnered 80 downloads
- API is deployed on Sageby Dev Server but connected to Sageby Live Database
- Backup on Sageby Dev Database
- Development Application is provisioned on 5 iPhones within the team
- Provisioned 1 iPhone that is with Sageby
Unit testing is a standard technique in computer programming whereby we break the software into small, independent pieces of code (a.k.a. the unit), isolate it from the rest of the code and test it. The main goal of unit testing is to try and find as many bugs as possible, as soon as possible.
Unit tests are split into two categories, Logic and Application Unit Tests.
Logic Unit Testing
Logic unit testing concerns the testing of the classes logic directly. You only test the logic of the methods, you don't interact with user interface elements or simulate them. For example, testing that a coordinate convertor utility is returning correct values is logic unit testing.
Percentage of Testing
- Verification of Credentials for manual login
- Test for verification of specific survey styles such as checkbox, radio buttons, etc.
- Test for mock data verification
Overall Percentage of logic tests : 50%
Application Unit Testing
Application testing in iOS is much more specific to applications with user interface. In application testing, you will check that clicking a certain button triggers the correct behavior, that a view loads correctly, and so on.
Percentage of Testing
- Landing page verification
- Survey, Rewards and Settings tab verification
- Survey details view and start button verification
- "About" button in settings tab verification
- "Share" button in settings tab verification
- "Logout" button in settings tab verification
Overall Percentage of Logic Tests : 70%
Test Case is a set of conditions or variables under which a we will determine whether an application or software system is working correctly or not. We have up to 50 steps for our Test Case version 3
Percentage of Testing
Covered Areas: All functional module
Overall Percentage of Test Case: 100%
The main objectives of this UT 3.1 are to gather receptiveness of our functionalities via Provision Phone Simulator.
- a. Login
- b. Survey
- c. Social Media Sharing
- d. Feedback (Setting)
This UT used a real survey called LTA Survey provided our client. They will do a real survey conducted on our phone, after which they will redeem a real physical voucher for us.
From our questionnaire, we asked about the learnability, satisfaction issues from the application and the highlighted functionalities. At the same time, they were encouraged to provide Heuristic Feedbacks based on specifications.
UT 3.1 Schedule
Altogether, there are 62 responses. Out of these 62, 34 are iPhone users and 23 are Non-iPhone. Also, there are 5 people who prefer not to state their phone model. Gender and age group is not considered in this test because LTA did not state any specific demographic preference.
Based on the 62 responses, they were asked on how easy it was it to do a survey using our application. 80% gave a 7 and above satisfactory rating based on a scale of 10.
When they were asked about their preferred social media platforms for sharing with their friends, 57 stated Facebook followed by 13 whom preferred Twitter. As users can select more than one checkbox, it will be important to focus on Facebook & Twitter as the preferred social media platform.
To gauge an understanding on how easy it was to learn how to use our application, 58% gave a full score out of 10. The remaining responses span from a scale of 6-10 (87%).
For an overall understanding of the satisfaction gain when using our application, 82% stated a 6 and above rating. Out of these 82%, 24% actually gave a full satisfactory score from a total score of 10.
In a nutshell, user reported about the slow connectivity while doing the survey. Also, some suggestions to the app were to provide a simple walkthrough and incentives to entice people so that they can find the motivation to share the app.
For heuristic wise, one of the glaring issues was a need to have a zoom feature so that user can see the survey question clearly. Other suggestions were a need for a more responsive survey form and include more colors if possible.
What are the main To-Do changes
- -Change Social Share content to Referral Link
- -Adding a Walkthrough feature at Iteration 6
The main objectives of UT 3.2 are to gather receptiveness of our functionalities via App Store Download Application.
- 1. Efficiency of Functionalities from Users via Appstore Download
- a. Login
- b. Survey
- c. Social Share
- d. Feedback
- 2. Heuristic Feedbacks (Designs)
- 3. Satisfaction Level and Learnability with Application
Furthermore, users are able to redeem a real physical McDonald's voucher after completion of a survey within the application.
UT 3.2 Schedule
Altogether, there were 42 responses. Their age group falls between 18 to 22 years old.
For A scenario, we have 20 people (48%) whereas for B scenario, we have 22 people (52%).
For User Phone model, 22 were using iPhone 4 (52%) while 13 were using iPhone 4S (31%). The remaining 7 people were not using iPhone.
As for iOS version, 13 were using iOS 5 (31%). Out of the 42, a huge majority of 62% were using iOS 5.1. Only 3 people were using iOS6.
A. Collecting Qualitative Data
Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes.
In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by the facilitators with instructions stated clearly. After which they were required to do an online questionnaire. It will be crafted around the usability issues and their satisfaction level of the application.
B. Collecting Quantitative Data
- Quantitative metrics to be measured include:
- -The amount of time to complete each task
Based on the 42 responses, 31% feel that the survey font visibility is just nice. However, 42% feel that it is larger than average. This is a preferred indicator.
As for the zoom feature, more than 90% preferred the zoom feature when doing the survey.
For doing the survey using our application, there were a mixes of responses based on a scale of 10 (1 being very difficult to 10 being very easy). 91% gave a rating for 4 and above while the remaining 9% gave a rating of 2 and below.
As for social sharing, 71% expressed sharing the application via Facebook/ Twitter while 26% indicated no interest in sharing with social platform.
Satisfaction wise, 74% expressed a rating of 7 and above. Overall, majority of the users are satisfied with using our application. For learnability, 98% gave a rating of 6 and above for learning to use our application. Generally, most do not find it difficult at all to use our application based on their first interaction.
- 1. Efficiency of Functionalities from Users via Appstore Download
- a. Login
- i. Mean: 1:00:20
- b. Survey
- i. Mean: 8:32:48
- c. Social Share
- i. Mean: 2:12:31
- d. Feedback
- i. Mean: 8:13:45
- a. Login
What are the main To-Do changes to our app
- • Landscape
- o Limitations of iPhone include the smaller screen size and hence landscape mode will only introduce a wider, but much shorter screen content
- • Enable popup when user clicks on tab bar while doing survey
- • Use smaller font that fills the whole screen & allow pinch to zoom
Our Team has gone through 4 iterations, from scoping, to planning, prototyping, constructing and carrying User Test.