HeaderSIS.jpg

IS480 Team wiki: 2012T1 Team Sageby UserTest4

From IS480
Jump to navigation Jump to search
Frontpage2.png

HOME

| PRODUCT OVERVIEW | PROJECT MANAGEMENT | USER TEST | LEARNING OUTCOMES


User Test 3.1

Objective


The main objectives of this UT 3.1 are to gather receptiveness of our functionalities via Provision Phone Simulator.

a. Login
b. Survey
c. Social Media Sharing
d. Feedback (Setting)


This UT used a real survey called LTA Survey provided our client. They will do a real survey conducted on our phone, after which they will redeem a real physical voucher for us.
From our questionnaire, we asked about the learnability, satisfaction issues from the application and the highlighted functionalities. At the same time, they were encouraged to provide Heuristic Feedbacks based on specifications.


UT 3.1 Schedule


CCCCC.GIF


User Profile


Altogether, there are 62 responses. Out of these 62, 34 are iPhone users and 23 are Non-iPhone. Also, there are 5 people who prefer not to state their phone model. Gender and age group is not considered in this test because LTA did not state any specific demographic preference.


Testing Methodology


Qualitative Metrics

Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. In addition, the facilitators are expected to look out for errors and navigation issues. Users will be doing by an online questionnaire only after finish using the application. It will be crafted around the usability issues and their satisfaction level of the application.


Repetition Matrix


Cccc.GIF


Analysis


Based on the 62 responses, they were asked on how easy it was it to do a survey using our application. 80% gave a 7 and above satisfactory rating based on a scale of 10.

When they were asked about their preferred social media platforms for sharing with their friends, 57 stated Facebook followed by 13 whom preferred Twitter. As users can select more than one checkbox, it will be important to focus on Facebook & Twitter as the preferred social media platform.

To gauge an understanding on how easy it was to learn how to use our application, 58% gave a full score out of 10. The remaining responses span from a scale of 6-10 (87%).

For an overall understanding of the satisfaction gain when using our application, 82% stated a 6 and above rating. Out of these 82%, 24% actually gave a full satisfactory score from a total score of 10.
In a nutshell, user reported about the slow connectivity while doing the survey. Also, some suggestions to the app were to provide a simple walkthrough and incentives to entice people so that they can find the motivation to share the app.

For heuristic wise, one of the glaring issues was a need to have a zoom feature so that user can see the survey question clearly. Other suggestions were a need for a more responsive survey form and include more colors if possible.


What are the main To-Do changes


-Change Social Share content to Referral Link
-Adding a Walkthrough feature at Iteration 6

What we did badly


Overall, the UT execution was executed with no major issues. There was no shortage of equipment or application failure. However, some of the suggestions that we could take note are the need to secure proper venue so that users can actually sit down and do the test under minimal distractions.

In addition, conducting the test on a basement was not desirable because phone connectivity was weak. It affects the time taken for loading and refreshing. Some of the UT users experiences experienced poor

What we did well


Proper instructions were given from the Testers to the Testees. All equipment’s are working fine. There were minimal waiting times. We exceeded our daily target for users.

User Test 3.2

Objective


The main objectives of UT 3.2 are to gather receptiveness of our functionalities via App Store Download Application.

1. Efficiency of Functionalities from Users via Appstore Download
a. Login
b. Survey
c. Social Share
d. Feedback
2. Heuristic Feedbacks (Designs)
3. Satisfaction Level and Learnability with Application


Furthermore, users are able to redeem a real physical McDonald's voucher after completion of a survey within the application.


UT 3.2 Schedule


Ee.GIF


User Profile


Altogether, there were 42 responses. Their age group falls between 18 to 22 years old.

For A scenario, we have 20 people (48%) whereas for B scenario, we have 22 people (52%).

For User Phone model, 22 were using iPhone 4 (52%) while 13 were using iPhone 4S (31%). The remaining 7 people were not using iPhone.

As for iOS version, 13 were using iOS 5 (31%). Out of the 42, a huge majority of 62% were using iOS 5.1. Only 3 people were using iOS6.

Testing Methodology


A. Collecting Qualitative Data
Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes.

In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by the facilitators with instructions stated clearly. After which they were required to do an online questionnaire. It will be crafted around the usability issues and their satisfaction level of the application.

B. Collecting Quantitative Data

Quantitative metrics to be measured include:
-The amount of time to complete each task


Repetition Matrix


Cccc.GIF


A/B Testing


Ff.GIF

Analysis


Survey
Based on the 42 responses, 31% feel that the survey font visibility is just nice. However, 42% feel that it is larger than average. This is a preferred indicator.

As for the zoom feature, more than 90% preferred the zoom feature when doing the survey.

For doing the survey using our application, there were a mixes of responses based on a scale of 10 (1 being very difficult to 10 being very easy). 91% gave a rating for 4 and above while the remaining 9% gave a rating of 2 and below.

As for social sharing, 71% expressed sharing the application via Facebook/ Twitter while 26% indicated no interest in sharing with social platform.

Satisfaction wise, 74% expressed a rating of 7 and above. Overall, majority of the users are satisfied with using our application. For learnability, 98% gave a rating of 6 and above for learning to use our application. Generally, most do not find it difficult at all to use our application based on their first interaction.

1. Efficiency of Functionalities from Users via Appstore Download
a. Login
i. Mean: 1:00:20
b. Survey
i. Mean: 8:32:48
c. Social Share
i. Mean: 2:12:31
d. Feedback
i. Mean: 8:13:45


What are the main To-Do changes to our app


• Enable popup when user clicks on tab bar while doing survey
• Use smaller font that fills the whole screen & allow pinch to zoom


What we did badly


Overall, the UT execution was executed with no major issues. There was no shortage of equipment or application failure. However, we do note that Testers can get a proper stopwatch instead of using phone as a stopwatch.

What we did well


Proper instructions were given from the Testers to the Testees. All equipment’s are working fine. There were minimal waiting times. We exceeded our daily target for test users on the first day.

Documentation

UserTest3.1 Schedule
UserTest3.1 Test Plan
UserTest3.1 Summary UserTest3.2 Schedule
UserTest3.2 Test Plan
UserTest3.2 Summary

User Test 4

Objective


1. Receptiveness of Functionalities from Users via Provision application a. Login ⁄ b. Survey ⁄ c. Redemption with QR Code ⁄
2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction)

3. Aesthetics of our application via Provisioned Phone Simulator.

UT 4 Schedule


Eff.GIF


User Profile


Altogether, there are 42 responses. Their age group range from 18 to 26. Majority of them were of age 21 (26%).

Gender wise, we have 23 males (55%) and 19 females (45%).

For internal validity purpose, 26 out 42 (62%) seated for our user test previously.

To understand user lifestyle usage, 55% were iPhone user while 38% were Android user.


Testing Methodology


Qualitative Metrics

Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by a online questionnaire with instructions stated clearly. It will be crafted around the usability issues and their satisfaction level of the application.


Repetition Matrix


Cccc.GIF



Analysis

1. Login

Issues:
- Long login process - Login failure when asked to permit application from posting on user’s behalf. (Error msg: Please update to latest version)
- Registered as user but not shown in the Profile. (Error msg: Please update to latest version)

2. Survey

Font Size
- 65% gave a neutral stand for the font size from our external survey (lime survey). 16% expressed that it was smaller than average while 17% stated that it was bigger than average.

Easiness
- 86% feel that it was easy to do the survey (7 and above). Out of the 86%, 62% gave a score of 8 and above.

Limesurvey issues
- Last question of ranking. Selected option does not tally with output selection.
- Submission of survey was not recorded and brought forward to a Limesurvey page.

Issues
- No credits accounted after completion of survey
- After clicking submit, error message “/www/..” appeared before success page comes out.

User Experience
- Satisfaction Level – 83% were pleased with the app. Score of 7 and above.
- Easiness to use the app – 86% finds it easy to use the app. Score of 7 and above.
- Spelling error such as “OKK” button
- Slow response time upon clicks

Improvement(s):
- Question size could fit whole iOS screen.
- Quality control the survey question and template. If not, it will look back on the app. (Sageby, take note).
- Add more color to the survey page
- Can add a redemption button after completion of survey.
- No horizontal scrolling

3. Redemption Module

Easiness for redemption
- 84% gave a score of 6 and above for redeem a reward using the application

Distinguishable between store and wallet function
- 73% find it okay.

Receptiveness of QR Code
- 63% find it exciting

Crash Alert
- Reward > Wallet > Ok > Crashed

Issues
- No credits accounted after completion of survey


Improvement:
- Display cumulative credits earned at the top right hand corner at every page.
- Grey out area that can’t be redeem
- Instruction Tab about the app.
- Highlight how store and wallet are related
- Show redemption history


4. Application User Experience

Satisfaction Level – 83% were pleased with the app. Score of 7 and above.
Easiness to use the app – 86% finds it easy to use the app. Score of 7 and above.

Issues
- Spelling error such as “OKK” button
- Slow response time upon clicks


Main Changes to App


Tff.GIF


What can be improve


The app was too buggy for user test. There was confusion from bug crashes from login process to redemption process. Store feature was buggy and hence, nothing were shown in wallet. There were lack of quality control for Limesurvey template and questions.

Test users did not arrive as per scheduled which causes unforeseen bottleneck and also no show occurrences.


What we did well


There was a proper demonstration of process flow via 3 booths for different purposes. Proper instructions were given from the Testers to the users. All equipment’s are working fine. There were minimal waiting times.


Documentation

User Test 4 Review Minutes
User Test 4 Schedule
User Test 4 Test Plan
User Test 4 Summary