HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T1 Team Sageby Midterm Wiki"

From IS480
Jump to navigation Jump to search
Line 526: Line 526:
 
Overall Percentage of logic tests : '''50%'''
 
Overall Percentage of logic tests : '''50%'''
  
======Sample Output======
+
======Output======
 
[[Image:Sageby Logic tests.png]]
 
[[Image:Sageby Logic tests.png]]
  

Revision as of 16:49, 30 September 2012

Frontpage2.png

HOME

| PRODUCT OVERVIEW | PROJECT MANAGEMENT | USER TEST | LEARNING OUTCOMES

Project Progress Summary

Project Summary

Since started off as a passionate team of 5 in May, we have gone through 4 iterations of construction, learning and verifications. We have strictly followed our core methodology, to Build, Measure and Learn that is adopted from Lean Startup. This methodology not only teaches us to change/pivot when necessary, but also to prevent wastage of efforts and time. Not only do we have to grab every opportunity to verify assumptions, build with leaner resources and learn from results, we have to understand client's needs and handle ever-changing situations.


What we have done thus far:

  • Built the application that can be used to do register with Sageby and create an account, do a survey and to share or feedback to Sageby Surveys
  • Constructed the Application Programming Interface that provides support for User and Survey data retrieval
  • Executed Beta Testing and Live Testing with more than 100 users combined
  • Carried a real survey by LTA that provides real rewards
  • Transitioned from Development Database to Live Database
  • Transitioned from iOS5 to iOS6
  • Deployed twice in Appstore and garnering 80 downloads till date

Project Highlights

Despite having a clear schedule and backed with good responses from Clients, there are still unseen situations that arises and needed our attention to either rectify the problem or grab the opportunity that improves our product.

1 Construction of Application Programming Interface (API)

To receive dynamic data and move beyond a Proof-Of-Concept product, Sageby graciously agreed to help to build the APIs that are necessary. However, their lead developer unforeseeably had to leave Sageby due to personal reasons.We had to make a drastic addition to our schedule by squeezing in time and manpower for building API while removing non-functional aspects such as testing for Performance Profiling and Memory Leaks.

Through this initiative, we managed to build the necessary APIs in time for our Live Test / User Test 3 that is an extremely important milestone in our project.

2 Transition from Development Database to Live Database

Sageby set out numerous stringent tests that we had to comply with before allowing us to access their live database. Thankfully, our diligence paid off and Sageby felt confident with having the application to access the live server.

However, before that, we had to clear several legacy issues such as MySQL plugin versioning issues, different PHP versions and unfortunately, face a database account user lockdown.

After ironing out issues and successful uploading of APIs, we managed to move from dummy data to real users data and this brought us to the next level in product development, from a Proof-Of-Concept to a usable and productive Application!

3 Live Testing with LTA Survey & Real Rewards

After carrying out the transitioning of database and inserting of API queries and codes, we managed to finish our iteration in time for User Test 3. However, Sageby proposed another idea that allows us to carry a real survey by LTA and to give out real rewards. Considering the fact that User Test 3 is targeted at a wider pool of users, we had to make changes to the current UT plan and accomodate the LTA survey.

This inclusion of LTA survey really showed us the true potential of the application as it shows that not only users are fine with doing surveys on mobile, many agreed that a niche type of survey can be dedicated for the mobile platform, Short Survey Polls.

Project Management

Project Timeline

Sageby ProjTimeline midterm.png

Schedule Brief

Sageby schedulebrief.png

Project Status

Task/function/features, etc Status Confident Level Comment
Login Module Fully deployed and tested 100%. However, it is not updated to latest Facebook SDK due to lack of backward compatibility with iOS 5 1 Luojia
Survey Module Fully deployed and tested 100%, with categorization via alphabetical order 1 Luojia
Rewards Module (Store) Fully deployed and tested 100%, with categorization via alphabetical order 1 Luojia
Rewards Module (Wallet) Fully deployed and tested 100%, with categorization via used & unused sections 1 Luojia
Settings Tab Fully deployed and tested 100% 1 JunHao & Mervyn
Social Sharing Module Fully deployed and tested 100% 1 TeeJun
Feedback Channel Fully deployed and tested 100%. Used Apptentive Package for feedbacks and easy rating portal. 1 Junhao
Performance Profiling & Memory Leak Tests Function to be Kept in View. To be continued if necessary. 1 Marcus
Performance Profiling & Memory Leak Tests Function to be Kept in View. To be continued if necessary. 0.5 Marcus
Application Programming Interface Partially developed, deployed and tested 50%. Survey, Users and Rewards Module completed 1 Marcus
Database Transition from Development to Live Fully deployed and tested 100% 0.8 Marcus
Unit Tests Partially developed, deployed and tested 50%. Login, Surveys, Rewards, User Module & Settings tab covered 0.8 JunHao
Facebook Sharing UIView Partially developed, deployed and tested 50%. 1 Mervyn
Mobile Template for Lime Survey Fully deployed and tested 100% 1 Mervyn & Junhao
Social Integration Partially developed, deployed and tested 30%. Twitter integration and Facebook Login 0.8 Luojia

Projet Schedule (Plan VS Actual)

Despite having several additions in our project scope, we managed to narrow down the non-functional requirements of the applications and added an extra week in Iteration 4 to accomodate our User Tests.

Planned Actual
Iteration Task Start End Start End Comment
1 Complete Paper Prototype for client's acceptance 30/5/12 5/6/12 30/5/12 5/6/12
Develop sample iOS App 6/6/12 13/6/12 6/6/12 13/6/12
Complete Prototype with Survey Module 14/6/12 27/6/12 14/6/12 27/6/12
2 Build Facebook Single Sign On Auth 27/6/12 3/7/12 27/6/12 3/7/12
Build Sageby Manual login 3/7/12 11/7/12 3/7/12 11/7/12
Allow user to logout 3/7/12 11/7/12 3/7/12 11/7/12
Build User details popup 11/7/12 17/7/12 11/7/12 17/7/12
Build different in-house survey layouts. Allow survey question to have dynamic survey layout 17/7/12 25/7/12 17/7/12 25/7/12
Able to pull dummy data from Sageby's server 17/7/12 25/7/12 17/7/12 25/7/12
Build survey's question routing function 17/7/12 25/7/12 17/7/12 25/7/12
Deploy and upload to Apple's Appstore 31/7/12 31/7/12 31/7/12 3/8/12 Code-signing error and outdated library links led to 3 days delay in deployment
3 Build browsable table for Store vouchers 1/8/12 8/8/12 1/8/12 8/8/12
Allow users to redeem vouchers 1/8/12 8/8/12 1/8/12 8/8/12
Build browsable table for Wallet vouchers 8/8/12 15/8/12 8/8/12 15/8/12
Build QR generator 8/8/12 15/8/12 8/8/12 15/8/12
Deployment on Appstore 21/8/12 21/8/12 8/9/12 8/9/12 Pushed to Iteration 3 due to the lack of APIs
4 User is able to share completed surveys via Facebook share and Twitter 22/8/12 29/8/12 22/8/12 29/8/12
Users will be able to provide feedbacks to Sageby Surveys 22/8/12 29/8/12 22/8/12 29/8/12
User is able to share referral link for benefits 22/8/12 28/8/12 22/8/12 28/8/12
Build iFrame to support Lime survey 29/8/12 4/9/12 29/8/12 4/9/12 Added an Extra week to support external survey engine
Build iFrame to support Lime survey (Added) 29/8/12 4/9/12 Needed to support external survey engine for live testing(UT3)
Built User & Survey Module's APIs (Added) 5/9/12 11/9/12 Added an Extra week to build APIs to receive dynamic data and prepare for live testing (UT3)
Transitioned from iOS5 to iOS6 (Added) 12/9/12 18/9/12 Added an Extra week to transition from iOS5 to iOS6 in order to update deprecated APIs and to support application on iPhone 5
Transitioned from Development Database to Live Database (Added) 12/9/12 18/9/12 User will be able to register with Sageby and do real surveys.
Deploy to Appstore 25/9/12 25/9/12 30/9/12 30/9/12 Delayed by 5 days due to rejection from Appstore for several unsupported APIs

Project Metrics

We have chosen Schedule Metric and Bug Metric as it best suits a development for iOS due to expected rapid changes in schedule and unforeseen bugginess of a mobile app.

Schedule Metric

Current Standing Sageby pm midterm.png

Issues Faced in Iteration 2

The team managed to complete new tasks on time in iteration 1. However, the technical complexity that we faced in iteration 2 was on a different level and we did not expect to face issues with customized survey styles and Facebook Login. This caused us to push our schedule back by 5 days.

Another issue was the delay of APIs from Sageby. This delay caused us to lose up to another 5 days in testing and development, and we had to hard code dummy data in order to meet the requirements for User test 2 for the iteration.

Issues Faced in Iteration 4

After Iteration 3, we had to re-scope iteration 4 due to several milestones to be met:

  • Move beyond proof of concept
  • Live Testing with real surveys and wide pool of users
  • Allow users to register with Sageby

In iteration 2, we faced numerous issues regarding unknown durations for new tasks. In iteration 4, we revised through these tasks and scope out based on the team's experience with development. We had to add 2 weeks to iteration 4, out of the 3 buffer weeks intended, in order to accomodate 3 other major phases, iOS6, Live Database and construction of APIs.

The schedule metric has been highly useful for us by providing us with an better grasp of our current situation and to track the amount of time we took with difficult tasks. Project Manager was also able to efficient allocate extra time and also to meet up with changing demands and opportunities such as the unexpected chance of deploying a real survey for UT3.

Bug Metric

Current Standing Sageby bug midterm.png

We needed the Bug Metric to answer 3 important questions during development:

  • Where did it originate from
  • Impact of the bug
  • How many are there

With the aid of the Bug Metric, we managed to identify key issues, which includes Facebook Login and sharing and persistent UI Bugs. Coding in Storyboard with Objective-C bypasses the conventional coding of transitions of Views in iOS. This is a huge improvement in convenience for new developers such as us, but when bugs arises, we need to identify the location of it and to rectify the issue by diving into the source codes and make manual changes.

Overall, the Bug Metric shows high correlation with the Schedule Metric and this is a relatively positive indication for the Project Manager that data collected responds well with the different Metrics in use.

Issues faced

Project Risks

We have split the risks into 2 components, Client and Project Management. Client's risk are rather intensive as Sageby is a new startup and they face plenty of risks that we have to take into account. Project Management on the other hand is rather standard but requires clear mitigation plan to avoid confusion within the team.

Client & Project Management

Sageby Risk1 midterm.png

Project Development

Sageby Risk2 midterm.png

Technical Complexity

Technical Complexity takes into consideration of Team Sageby's main development pain points. Although there are other complexities such as Facebook login SDK issues and iOS 6 with backwards compatibility, these issues can be temporarily kept in view due to the immaturity of the SDK and platforms.

Sageby Technical complexity midterm.png

Quality of Product

Sageby Surveys formally launched on 18th of September and it comes with a full fledged working survey module which allows users registration with Sageby. The quality of product mentioned below focuses more on the non-functional aspects of the Application as functional tasks list are stated in the project scope. Also, these deliverables are constitutes the structure of the application and ensure its integrity and development modularity.

Sageby Qualityofproduct midterm.png

Intermediate Deliverables

Stage Specification Modules
Project Management Meeting Minutes Team meetings: Minutes 1-8
Supervisor: Minutes 1-3
Project Schedule Planning Project Schedule: Project Schedule
Metric Bug Metric: Bug Metric
Schedule Metric: Schedule Metric
Requirements Mid Term Presentation To-Be-Uploaded
Analysis Use case Use Case Diagram & Documentation
Functionality List Functionality List
Design Architecture System Diagram System Architecture
Mockup Designs Mockup Designs
User Test Documents UT 1

UserTest1 Schedule
UserTest1 Test Plan
UserTest1 Summary

UT 2

UserTest2 Schedule
UserTest2 Test Plan
UserTest2 Summary

UT 3

UserTest3.1 Schedule
UserTest3.1 Test Plan
UserTest3.1 Summary

UserTest3.2 Schedule
UserTest3.2 Test Plan
UserTest3.2 Summary

Deployment

  • Sageby Surveys Version 1.1 is currently deployed on Appstore
    • Garnered 80 downloads
  • API is deployed on Sageby Dev Server but connected to Sageby Live Database
    • Backup on Sageby Dev Database
  • Development Application is provisioned on 5 iPhones within the team
    • Provisioned 1 iPhone that is with Sageby

Testing

Unit Testing

Objective

Unit testing is a standard technique in computer programming whereby we break the software into small, independent pieces of code (a.k.a. the unit), isolate it from the rest of the code and test it. The main goal of unit testing is to try and find as many bugs as possible, as soon as possible.

Unit tests are split into two categories, Logic and Application Unit Tests.

Logic Unit Testing
Objective

Logic unit testing concerns the testing of the classes logic directly. You only test the logic of the methods, you don't interact with user interface elements or simulate them. For example, testing that a coordinate convertor utility is returning correct values is logic unit testing.

Percentage of Testing

Covered Areas:

  • Verification of Credentials for manual login
  • Test for verification of specific survey styles such as checkbox, radio buttons, etc.
  • Test for mock data verification

Overall Percentage of logic tests : 50%

Output

Sageby Logic tests.png

Application Unit Testing
Objective

Application testing in iOS is much more specific to applications with user interface. In application testing, you will check that clicking a certain button triggers the correct behavior, that a view loads correctly, and so on.

Percentage of Testing

Covered Areas:

  • Landing page verification
  • Survey, Rewards and Settings tab verification
  • Survey details view and start button verification
  • "About" button in settings tab verification
  • "Share" button in settings tab verification
  • "Logout" button in settings tab verification

Overall Percentage of Logic Tests : 70%

Output

Sageby Application test.png

Test Cases

Objective

Test Case is a set of conditions or variables under which a we will determine whether an application or software system is working correctly or not. We have up to 50 steps for our Test Case version 3

Percentage of Testing

Covered Areas: All functional module

Overall Percentage of Test Case: 100%

User Test

UT 3.1

Objective

The main objectives of this UT 3.1 are to gather receptiveness of our functionalities via Provision Phone Simulator.

a. Login
b. Survey
c. Social Media Sharing
d. Feedback (Setting)


This UT used a real survey called LTA Survey provided our client. They will do a real survey conducted on our phone, after which they will redeem a real physical voucher for us.
From our questionnaire, we asked about the learnability, satisfaction issues from the application and the highlighted functionalities. At the same time, they were encouraged to provide Heuristic Feedbacks based on specifications.


UT 3.1 Schedule

CCCCC.GIF


User Profile


Altogether, there are 62 responses. Out of these 62, 34 are iPhone users and 23 are Non-iPhone. Also, there are 5 people who prefer not to state their phone model. Gender and age group is not considered in this test because LTA did not state any specific demographic preference.


Testing Methodology

Qualitative Metrics

Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. In addition, the facilitators are expected to look out for errors and navigation issues. Users will be doing by an online questionnaire only after finish using the application. It will be crafted around the usability issues and their satisfaction level of the application.


Repetition Matrix

Cccc.GIF


Analysis


Based on the 62 responses, they were asked on how easy it was it to do a survey using our application. 80% gave a 7 and above satisfactory rating based on a scale of 10.

When they were asked about their preferred social media platforms for sharing with their friends, 57 stated Facebook followed by 13 whom preferred Twitter. As users can select more than one checkbox, it will be important to focus on Facebook & Twitter as the preferred social media platform.

To gauge an understanding on how easy it was to learn how to use our application, 58% gave a full score out of 10. The remaining responses span from a scale of 6-10 (87%).

For an overall understanding of the satisfaction gain when using our application, 82% stated a 6 and above rating. Out of these 82%, 24% actually gave a full satisfactory score from a total score of 10.
In a nutshell, user reported about the slow connectivity while doing the survey. Also, some suggestions to the app were to provide a simple walkthrough and incentives to entice people so that they can find the motivation to share the app.

For heuristic wise, one of the glaring issues was a need to have a zoom feature so that user can see the survey question clearly. Other suggestions were a need for a more responsive survey form and include more colors if possible.


What are the main To-Do changes


-Change Social Share content to Referral Link
-Adding a Walkthrough feature at Iteration 6

UT 3.2

Objective

The main objectives of UT 3.2 are to gather receptiveness of our functionalities via App Store Download Application.

1. Efficiency of Functionalities from Users via Appstore Download
a. Login
b. Survey
c. Social Share
d. Feedback
2. Heuristic Feedbacks (Designs)
3. Satisfaction Level and Learnability with Application


Furthermore, users are able to redeem a real physical McDonald's voucher after completion of a survey within the application.


UT 3.2 Schedule

Ee.GIF


User Profile


Altogether, there were 42 responses. Their age group falls between 18 to 22 years old.

For A scenario, we have 20 people (48%) whereas for B scenario, we have 22 people (52%).

For User Phone model, 22 were using iPhone 4 (52%) while 13 were using iPhone 4S (31%). The remaining 7 people were not using iPhone.

As for iOS version, 13 were using iOS 5 (31%). Out of the 42, a huge majority of 62% were using iOS 5.1. Only 3 people were using iOS6.

Testing Methodology


A. Collecting Qualitative Data
Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes.

In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by the facilitators with instructions stated clearly. After which they were required to do an online questionnaire. It will be crafted around the usability issues and their satisfaction level of the application.

B. Collecting Quantitative Data

Quantitative metrics to be measured include:
-The amount of time to complete each task


Repetition Matrix

Cccc.GIF


A/B Testing

Ff.GIF
Analysis


Survey
Based on the 42 responses, 31% feel that the survey font visibility is just nice. However, 42% feel that it is larger than average. This is a preferred indicator.

As for the zoom feature, more than 90% preferred the zoom feature when doing the survey.

For doing the survey using our application, there were a mixes of responses based on a scale of 10 (1 being very difficult to 10 being very easy). 91% gave a rating for 4 and above while the remaining 9% gave a rating of 2 and below.

As for social sharing, 71% expressed sharing the application via Facebook/ Twitter while 26% indicated no interest in sharing with social platform.

Satisfaction wise, 74% expressed a rating of 7 and above. Overall, majority of the users are satisfied with using our application. For learnability, 98% gave a rating of 6 and above for learning to use our application. Generally, most do not find it difficult at all to use our application based on their first interaction.

1. Efficiency of Functionalities from Users via Appstore Download
a. Login
i. Mean: 1:00:20
b. Survey
i. Mean: 8:32:48
c. Social Share
i. Mean: 2:12:31
d. Feedback
i. Mean: 8:13:45


What are the main To-Do changes to our app


• Landscape
o Limitations of iPhone include the smaller screen size and hence landscape mode will only introduce a wider, but much shorter screen content
• Enable popup when user clicks on tab bar while doing survey
• Use smaller font that fills the whole screen & allow pinch to zoom


Reflection

Our Team has gone through 4 iterations, from scoping, to planning, prototyping, constructing and carrying User Test.

Team Reflection