HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T1 Team Sageby UserTest4"

From IS480
Jump to navigation Jump to search
 
(20 intermediate revisions by 2 users not shown)
Line 31: Line 31:
 
| style="border-bottom:1px solid #F7F7F7; border-top:1px solid #F7F7F7; background:none;"  |<font color="#545454"><b>|</b></font>
 
| style="border-bottom:1px solid #F7F7F7; border-top:1px solid #F7F7F7; background:none;"  |<font color="#545454"><b>|</b></font>
  
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest1 |<font color="#5C61A1"><b>USER TEST 1</b></font>]]
+
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest1 |<font color="#5C61A1"><b>PROTOTYPE <br>USER TEST</b></font>]]
 
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |<font color="#545454"><b>|</b></font>  
 
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |<font color="#545454"><b>|</b></font>  
  
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest2 |<font color="#5C61A1"><b>USER TEST 2</b></font>]]
+
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest4 |<font color="#545454"><b>BETA TEST</b></font>]]
 
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |<font color="#545454"><b>|</b></font>  
 
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |<font color="#545454"><b>|</b></font>  
  
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest3 |<font color="#5C61A1"><b>USER TEST 3</b></font>]]
+
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest5 |<font color="#5C61A1"><b>FINAL USER TEST</b></font>]]
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |<font color="#545454"><b>|</b></font>  
+
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |
 +
|}
 +
</div>
 +
<br>
  
| style="padding:0 .4em; background-color:#F7F7F7; font-family:Helvetica;  border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; text-align:center;  color:#000000" width="16%" |[[IS480_Team_wiki:_2012T1_Team_Sageby_UserTest4 |<font color="#545454"><b>USER TEST 4</b></font>]]
+
=User Test 3.1=
| style="border-bottom:5px solid #F7F7F7; border-top:5px solid #F7F7F7; background:none;" |<font color="#545454"><b>|</b></font>  
+
==Objective==
|}
+
<div class="left"><br>
 +
The main objectives of this UT 3.1 are to gather receptiveness of our functionalities via Provision Phone Simulator.
 +
:a. Login
 +
:b. Survey
 +
:c. Social Media Sharing
 +
:d. Feedback (Setting)
 +
<br>This UT used a real survey called LTA Survey provided our client. They will do a real survey conducted on our phone, after which they will redeem a real physical voucher for us.
 +
<br>From our questionnaire, we asked about the learnability, satisfaction issues from the application and the highlighted functionalities. At the same time, they were encouraged to provide Heuristic Feedbacks based on specifications.
 +
</div>
 +
<br>
 +
== UT 3.1 Schedule ==
 +
<div class="center"><br>
 +
[[Image:CCCCC.GIF]]</div><br>
 +
== User Profile ==
 +
<div class="left">
 +
<br>Altogether, there are 62 responses. Out of these 62, 34 are iPhone users and 23 are Non-iPhone. Also, there are 5 people who prefer not to state their phone model. Gender and age group is not considered in this test because LTA did not state any specific demographic preference.
 
</div>
 
</div>
 
<br>
 
<br>
= Objective =
+
== Testing Methodology ==
 +
<div class="left"><br>
 +
'''Qualitative Metrics'''<br>
 +
Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. In addition, the facilitators are expected to look out for errors and navigation issues. Users will be doing by an online questionnaire only after finish using the application. It will be crafted around the usability issues and their satisfaction level of the application.</div><br>
 +
== Repetition Matrix ==
 +
<div class="center"><br>
 +
[[Image:cccc.GIF]]</div><br>
 +
== Analysis ==
 +
<div class="left">
 +
<br>Based on the 62 responses, they were asked on how easy it was it to do a survey using our application. 80% gave a 7 and above satisfactory rating based on a scale of 10.
 +
<br><br>When they were asked about their preferred social media platforms for sharing with their friends, 57 stated Facebook followed by 13 whom preferred Twitter. As users can select more than one checkbox, it will be important to focus on Facebook & Twitter as the preferred social media platform.
 +
<br><br>To gauge an understanding on how easy it was to learn how to use our application, 58% gave a full score out of 10.  The remaining responses span from a scale of 6-10 (87%).
 +
<br><br>For an overall understanding of the satisfaction gain when using our application, 82% stated a 6 and above rating. Out of these 82%, 24% actually gave a full satisfactory score from a total score of 10.
 +
<br>In a nutshell, user reported about the slow connectivity while doing the survey. Also, some suggestions to the app were to provide a simple walkthrough and incentives to entice people so that they can find the motivation to share the app.
 +
<br><br>For heuristic wise, one of the glaring issues was a need to have a zoom feature so that user can see the survey question clearly. Other suggestions were a need for a more responsive survey form and include more colors if possible.
 +
</div><br>
 +
== What are the main To-Do changes ==
 +
<br>
 +
:-Change Social Share content to Referral Link
 +
:-Adding a Walkthrough feature at Iteration 6<br>
 +
== What we did badly ==
 +
<br>Overall, the UT execution was executed with no major issues. There was no shortage of equipment or application failure. However, some of the suggestions that we could take note are the need to secure proper venue so that users can actually sit down and do the test under minimal distractions.
 +
<br><br>In addition, conducting the test on a basement was not desirable because phone connectivity was weak. It affects the time taken for loading and refreshing. Some of the UT users experiences experienced poor
 +
<br>
 +
== What we did well ==
 +
<br>Proper instructions were given from the Testers to the Testees. All equipment’s are working fine. There were minimal waiting times. We exceeded our daily target for users. <br>
 +
<br>
 +
 
 +
=User Test 3.2=
 +
== Objective ==
 +
<div class="left"><br>
 +
The main objectives of UT 3.2 are to gather receptiveness of our functionalities via App Store Download Application.
 +
:1. Efficiency of Functionalities from Users via Appstore Download
 +
::a. Login
 +
::b. Survey
 +
::c. Social Share
 +
::d. Feedback
 +
:2. Heuristic Feedbacks (Designs)
 +
:3. Satisfaction Level and Learnability with Application
 +
 
 +
<br>Furthermore, users are able to redeem a real physical McDonald's voucher after completion of a survey within the application.
 +
</div><br>
 +
== UT 3.2 Schedule ==
 +
<div class="center"><br>
 +
[[Image:ee.GIF]]</div><br>
 +
== User Profile ==
 +
<br>Altogether, there were 42 responses. Their age group falls between 18 to 22 years old.
 +
<br><br>For A scenario, we have 20 people (48%) whereas for B scenario, we have 22 people (52%).
 +
<br><br>For User Phone model, 22 were using iPhone 4 (52%) while 13 were using iPhone 4S (31%). The remaining 7 people were not using iPhone.
 +
<br><br>As for iOS version, 13 were using iOS 5 (31%). Out of the 42, a huge majority of 62% were using iOS 5.1. Only 3 people were using iOS6.
 +
<br>
 +
== Testing Methodology ==
 +
<br>
 +
'''A. Collecting Qualitative Data'''<br>
 +
Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. <br><br>In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by the facilitators with instructions stated clearly. After which they were required to do an online questionnaire. It will be crafted around the usability issues and their satisfaction level of the application.<br>
 +
<br>
 +
'''B. Collecting Quantitative Data'''<br>
 +
:Quantitative metrics to be measured include:
 +
::-The amount of time to complete each task
 +
<br>
 +
== Repetition Matrix ==
 +
<div class="center"><br>
 +
[[Image:cccc.GIF]]</div><br>
 +
== A/B Testing ==
 +
<div class="center"><br>[[Image:ff.GIF]]</div>
 +
== Analysis ==
 +
<br>
 +
 
 +
'''Survey'''
 +
<br>Based on the 42 responses, 31% feel that the survey font visibility is just nice. However, 42% feel that it is larger than average. This is a preferred indicator.
 +
<br><br>As for the zoom feature, more than 90% preferred the zoom feature when doing the survey.
 +
<br><br>For doing the survey using our application, there were a mixes of responses based on a scale of 10 (1 being very difficult to 10 being very easy). 91% gave a rating for 4 and above while the remaining 9% gave a rating of 2 and below.
 +
<br><br>As for social sharing, 71% expressed sharing the application via Facebook/ Twitter while 26% indicated no interest in sharing with social platform.
 +
<br><br>Satisfaction wise, 74% expressed a rating of 7 and above. Overall, majority of the users are satisfied with using our application. For learnability, 98% gave a rating of 6 and above for learning to use our application. Generally, most do not find it difficult at all to use our application based on their first interaction.
 +
<br><br>
 +
:1. Efficiency of Functionalities from Users via Appstore Download
 +
::a. Login
 +
:::i. Mean: 1:00:20
 +
::b. Survey
 +
:::i. Mean: 8:32:48
 +
::c. Social Share
 +
:::i. Mean: 2:12:31
 +
::d. Feedback
 +
:::i. Mean: 8:13:45
 +
<br>
 +
== What are the main To-Do changes to our app ==
 +
<br>
 +
:• Enable popup when user clicks on tab bar while doing survey
 +
:• Use smaller font that fills the whole screen & allow pinch to zoom
 +
<br>
 +
 
 +
== What we did badly ==
 +
<br>Overall, the UT execution was executed with no major issues. There was no shortage of equipment or application failure. However, we do note that Testers can get a proper stopwatch instead of using phone as a stopwatch.<br>
 +
== What we did well ==
 +
<br>Proper instructions were given from the Testers to the Testees. All equipment’s are working fine. There were minimal waiting times. We exceeded our daily target for test users on the first day.<br>
 +
 
 +
==Documentation==
 +
[[Media:Sageby UT 3.1 Schedule.docx|UserTest3.1 Schedule]]<br />
 +
[[Media:Sageby UT 3.1 Test Plan.docx|UserTest3.1 Test Plan]]<br />
 +
[[Media:Sageby UT 3.1 Summary.docx|UserTest3.1 Summary]]
 +
[[Media:Sageby UT 3.2 Schedule.docx|UserTest3.2 Schedule]]<br />
 +
[[Media:Sageby UT 3.2 Test Plan.docx|UserTest3.2 Test Plan]]<br />
 +
[[Media:Sageby UT 3.2 Summary.docx|UserTest3.2 Summary]]
 +
 
 +
=User Test 4=
 +
== Objective ==
 
<div class="left">
 
<div class="left">
 
<br>1. Receptiveness of Functionalities from Users via Provision application
 
<br>1. Receptiveness of Functionalities from Users via Provision application
Line 54: Line 177:
 
3. Aesthetics of our application via Provisioned Phone Simulator.</div>
 
3. Aesthetics of our application via Provisioned Phone Simulator.</div>
  
=UT 4 Schedule=
+
==UT 4 Schedule==
 
<div class="center"><br>
 
<div class="center"><br>
 
[[Image:eff.GIF]]</div><br>
 
[[Image:eff.GIF]]</div><br>
  
= User Profile =
+
== User Profile ==
 
<div class="left">
 
<div class="left">
 
<br>Altogether, there are 42 responses. Their age group range from 18 to 26. Majority of them were of age 21 (26%).
 
<br>Altogether, there are 42 responses. Their age group range from 18 to 26. Majority of them were of age 21 (26%).
Line 67: Line 190:
 
<br>
 
<br>
  
= Testing Methodology =
+
==Testing Methodology==
 
<div class="left">
 
<div class="left">
 
<br>'''Qualitative Metrics'''<br>
 
<br>'''Qualitative Metrics'''<br>
Line 78: Line 201:
  
  
= Analysis =
+
== Analysis ==
 
<div class="left">
 
<div class="left">
 +
<b>1. Login</b>
 +
<br><br><b>Issues:</b>
 +
<br>- Long login process
 +
- Login failure when asked to permit application from posting on user’s behalf. (Error msg: Please update to latest version)
 +
<br>- Registered as user but not shown in the Profile. (Error msg: Please update to latest version)
 
<br>
 
<br>
:-100% find it easy to do a survey using our application, with a rating of 7 and above.
+
<br><b>2. Survey</b>
:-64% prefer clicking the “Next” Button manually
+
<br><br><b>Font Size</b>
:-100% faced no difficulties identifying & purchasing the voucher, :-with a rating of 6 and above.
+
<br>- 65% gave a neutral stand for the font size from our external survey (lime survey). 16% expressed that it was smaller than average while 17% stated that it was bigger than average.
:-90% find it easy to learn how to use our application.
+
<br><br><b>Easiness</b>
:-90% gave a satisfactory response from using our application.
+
<br>- 86% feel that it was easy to do the survey (7 and above). Out of the 86%, 62% gave a score of 8 and above.  
</div><br>
+
<br><br>
= Main Changes to App =
+
<b>Limesurvey issues</b>
<div class="left"><br>
+
<br> - Last question of ranking. Selected option does not tally with output selection.
 +
<br> - Submission of survey was not recorded and brought forward to a Limesurvey page.
 +
<br><br>
 +
<b>Issues</b>
 +
<br> - No credits accounted after completion of survey
 +
<br> - After clicking submit, error message “/www/..” appeared before success page comes out.
 +
<br><br>
 +
<b>User Experience</b>
 +
<br> - Satisfaction Level – 83% were pleased with the app. Score of 7 and above.
 +
<br> - Easiness to use the app – 86% finds it easy to use the app. Score of 7 and above.
 +
<br> - Spelling error such as “OKK” button
 +
<br> - Slow response time upon clicks
 +
<br><br><b>Improvement(s):</b>
 +
<br>- Question size could fit whole iOS screen.
 +
<br>- Quality control the survey question and template. If not, it will look back on the app. (Sageby, take note).
 +
<br>- Add more color to the survey page
 +
<br>- Can add a redemption button after completion of survey.
 +
<br>- No horizontal scrolling
 +
<br>
 +
<br><b>3. Redemption Module</b>
 +
<br><br><b>Easiness for redemption</b>
 +
<br>- 84% gave a score of 6 and above for redeem a reward using the application
 +
<br><br><b>Distinguishable between store and wallet function</b>
 +
<br>- 73% find it okay.
 +
<br><br><b>Receptiveness of QR Code</b>
 +
<br>- 63% find it exciting
 +
<br><br><b>Crash Alert</b>
 +
<br>- Reward > Wallet > Ok > Crashed
 +
<br><br><b>Issues</b>
 +
<br>- No credits accounted after completion of survey
  
'''Resolved know bugs'''
+
<br><b>Improvement:</b>
:Size of Survey questions options
+
<br>- Display cumulative credits earned at the top right hand corner at every page.
:Fixing 2nd survey crashing issue
+
<br>- Grey out area that can’t be redeem
 +
<br>- Instruction Tab about the app.
 +
<br>- Highlight how store and wallet are related
 +
<br>- Show redemption history
 
<br>
 
<br>
'''Heuristic'''
+
<br><br><b>4. Application User Experience</b>
:Flow Bar while completing surveys
+
<br><br>Satisfaction Level – 83% were pleased with the app. Score of 7 and above.
<br>
+
<br>Easiness to use the app – 86% finds it easy to use the app. Score of 7 and above.
'''Functionalities'''
+
<br><br><b>Issues</b>
:Change "Do more surveys" to "Back to Surveys"
+
<br>- Spelling error such as “OKK” button
:Adding a popup to confirm purchase
+
<br>- Slow response time upon clicks
:Arrange Vouchers in alphabetical order
+
 
:Fix up API to check QR Code validity
 
:Split the Redeemed vouchers into 2 categories, redeemed and :available.
 
 
</div><br>
 
</div><br>
= What we did badly =
+
 
<div class="left"><br>Overall, the UT execution faces no major issues pertaining to equipment lapse or application failure. However, from an operation perspective, few UT users failed to turn up for the test on time. It could be due to various reasons.
+
== Main Changes to App ==
Secondly, some UT users did not have the right frame of mind while doing the test. It could affect the quality of the feedbacks.
+
<div class="center"><br>
Thirdly, UT users experienced difficulties understanding the objectives of the UT and couldn’t handle the application well. Reasons could be the lack of clear explanation or users could be misguided
+
[[Image:tff.GIF]]</div><br>
 +
 
 +
== What can be improve ==
 +
<div class="left"><br>The app was too buggy for user test. There was confusion from bug crashes from login process to redemption process. Store feature was buggy and hence, nothing were shown in wallet. There were lack of quality control for Limesurvey template and questions.  
 +
Test users did not arrive as per scheduled which causes unforeseen bottleneck and also no show occurrences.
 +
 
 
</div><br>
 
</div><br>
= What we did well =
+
 
<div class="left"><br>
+
==What we did well==
Users did not find a problem locating the test venue. Test equipment and application are running fine.</div><br>
 
= What to improve for future UTs =
 
 
<div class="left"><br>
 
<div class="left"><br>
:1) Reminders to ensure user attendance
+
There was a proper demonstration of process flow via 3 booths for different purposes. Proper instructions were given from the Testers to the users. All equipment’s are working fine. There were minimal waiting times. </div><br>
:2) Quality control the UT users, making sure its fall under the targeted demographics.
 
:3) Standardized opening speech to all users to ensure that they know the objectives of the UT.</div>
 
  
=Documentation=
+
==Documentation==
[[Media:Sageby UT 2 Schedule.docx|UserTest2 Schedule]]<br />
+
[[Media:Post UT 4 Review Minutes.docx|User Test  4 Review Minutes]]<br />
[[Media:Sageby UT 2 Test Plan.docx|UserTest2 Test Plan]]<br />
+
[[Media:UT 4 Schedule.docx|User Test 4 Schedule]]<br />
[[Media:Sageby UT 2 Summary.docx|UserTest2 Summary]]
+
[[Media:UT 4 Test Plan.docx|User Test 4 Test Plan]]<br />
 +
[[Media:UT 4 Summary.docx|User Test 4 Summary]]

Latest revision as of 16:26, 25 November 2012

Frontpage2.png

HOME

| PRODUCT OVERVIEW | PROJECT MANAGEMENT | USER TEST | LEARNING OUTCOMES


User Test 3.1

Objective


The main objectives of this UT 3.1 are to gather receptiveness of our functionalities via Provision Phone Simulator.

a. Login
b. Survey
c. Social Media Sharing
d. Feedback (Setting)


This UT used a real survey called LTA Survey provided our client. They will do a real survey conducted on our phone, after which they will redeem a real physical voucher for us.
From our questionnaire, we asked about the learnability, satisfaction issues from the application and the highlighted functionalities. At the same time, they were encouraged to provide Heuristic Feedbacks based on specifications.


UT 3.1 Schedule


CCCCC.GIF


User Profile


Altogether, there are 62 responses. Out of these 62, 34 are iPhone users and 23 are Non-iPhone. Also, there are 5 people who prefer not to state their phone model. Gender and age group is not considered in this test because LTA did not state any specific demographic preference.


Testing Methodology


Qualitative Metrics

Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. In addition, the facilitators are expected to look out for errors and navigation issues. Users will be doing by an online questionnaire only after finish using the application. It will be crafted around the usability issues and their satisfaction level of the application.


Repetition Matrix


Cccc.GIF


Analysis


Based on the 62 responses, they were asked on how easy it was it to do a survey using our application. 80% gave a 7 and above satisfactory rating based on a scale of 10.

When they were asked about their preferred social media platforms for sharing with their friends, 57 stated Facebook followed by 13 whom preferred Twitter. As users can select more than one checkbox, it will be important to focus on Facebook & Twitter as the preferred social media platform.

To gauge an understanding on how easy it was to learn how to use our application, 58% gave a full score out of 10. The remaining responses span from a scale of 6-10 (87%).

For an overall understanding of the satisfaction gain when using our application, 82% stated a 6 and above rating. Out of these 82%, 24% actually gave a full satisfactory score from a total score of 10.
In a nutshell, user reported about the slow connectivity while doing the survey. Also, some suggestions to the app were to provide a simple walkthrough and incentives to entice people so that they can find the motivation to share the app.

For heuristic wise, one of the glaring issues was a need to have a zoom feature so that user can see the survey question clearly. Other suggestions were a need for a more responsive survey form and include more colors if possible.


What are the main To-Do changes


-Change Social Share content to Referral Link
-Adding a Walkthrough feature at Iteration 6

What we did badly


Overall, the UT execution was executed with no major issues. There was no shortage of equipment or application failure. However, some of the suggestions that we could take note are the need to secure proper venue so that users can actually sit down and do the test under minimal distractions.

In addition, conducting the test on a basement was not desirable because phone connectivity was weak. It affects the time taken for loading and refreshing. Some of the UT users experiences experienced poor

What we did well


Proper instructions were given from the Testers to the Testees. All equipment’s are working fine. There were minimal waiting times. We exceeded our daily target for users.

User Test 3.2

Objective


The main objectives of UT 3.2 are to gather receptiveness of our functionalities via App Store Download Application.

1. Efficiency of Functionalities from Users via Appstore Download
a. Login
b. Survey
c. Social Share
d. Feedback
2. Heuristic Feedbacks (Designs)
3. Satisfaction Level and Learnability with Application


Furthermore, users are able to redeem a real physical McDonald's voucher after completion of a survey within the application.


UT 3.2 Schedule


Ee.GIF


User Profile


Altogether, there were 42 responses. Their age group falls between 18 to 22 years old.

For A scenario, we have 20 people (48%) whereas for B scenario, we have 22 people (52%).

For User Phone model, 22 were using iPhone 4 (52%) while 13 were using iPhone 4S (31%). The remaining 7 people were not using iPhone.

As for iOS version, 13 were using iOS 5 (31%). Out of the 42, a huge majority of 62% were using iOS 5.1. Only 3 people were using iOS6.

Testing Methodology


A. Collecting Qualitative Data
Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes.

In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by the facilitators with instructions stated clearly. After which they were required to do an online questionnaire. It will be crafted around the usability issues and their satisfaction level of the application.

B. Collecting Quantitative Data

Quantitative metrics to be measured include:
-The amount of time to complete each task


Repetition Matrix


Cccc.GIF


A/B Testing


Ff.GIF

Analysis


Survey
Based on the 42 responses, 31% feel that the survey font visibility is just nice. However, 42% feel that it is larger than average. This is a preferred indicator.

As for the zoom feature, more than 90% preferred the zoom feature when doing the survey.

For doing the survey using our application, there were a mixes of responses based on a scale of 10 (1 being very difficult to 10 being very easy). 91% gave a rating for 4 and above while the remaining 9% gave a rating of 2 and below.

As for social sharing, 71% expressed sharing the application via Facebook/ Twitter while 26% indicated no interest in sharing with social platform.

Satisfaction wise, 74% expressed a rating of 7 and above. Overall, majority of the users are satisfied with using our application. For learnability, 98% gave a rating of 6 and above for learning to use our application. Generally, most do not find it difficult at all to use our application based on their first interaction.

1. Efficiency of Functionalities from Users via Appstore Download
a. Login
i. Mean: 1:00:20
b. Survey
i. Mean: 8:32:48
c. Social Share
i. Mean: 2:12:31
d. Feedback
i. Mean: 8:13:45


What are the main To-Do changes to our app


• Enable popup when user clicks on tab bar while doing survey
• Use smaller font that fills the whole screen & allow pinch to zoom


What we did badly


Overall, the UT execution was executed with no major issues. There was no shortage of equipment or application failure. However, we do note that Testers can get a proper stopwatch instead of using phone as a stopwatch.

What we did well


Proper instructions were given from the Testers to the Testees. All equipment’s are working fine. There were minimal waiting times. We exceeded our daily target for test users on the first day.

Documentation

UserTest3.1 Schedule
UserTest3.1 Test Plan
UserTest3.1 Summary UserTest3.2 Schedule
UserTest3.2 Test Plan
UserTest3.2 Summary

User Test 4

Objective


1. Receptiveness of Functionalities from Users via Provision application a. Login ⁄ b. Survey ⁄ c. Redemption with QR Code ⁄
2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction)

3. Aesthetics of our application via Provisioned Phone Simulator.

UT 4 Schedule


Eff.GIF


User Profile


Altogether, there are 42 responses. Their age group range from 18 to 26. Majority of them were of age 21 (26%).

Gender wise, we have 23 males (55%) and 19 females (45%).

For internal validity purpose, 26 out 42 (62%) seated for our user test previously.

To understand user lifestyle usage, 55% were iPhone user while 38% were Android user.


Testing Methodology


Qualitative Metrics

Users are allowed to express their opinions while accomplishing each task. The intention is allow the facilitators to record observations and important details of the users' thought processes. In addition, the facilitators are expected to look out for errors and navigation issues. Users will be guided by a online questionnaire with instructions stated clearly. It will be crafted around the usability issues and their satisfaction level of the application.


Repetition Matrix


Cccc.GIF



Analysis

1. Login

Issues:
- Long login process - Login failure when asked to permit application from posting on user’s behalf. (Error msg: Please update to latest version)
- Registered as user but not shown in the Profile. (Error msg: Please update to latest version)

2. Survey

Font Size
- 65% gave a neutral stand for the font size from our external survey (lime survey). 16% expressed that it was smaller than average while 17% stated that it was bigger than average.

Easiness
- 86% feel that it was easy to do the survey (7 and above). Out of the 86%, 62% gave a score of 8 and above.

Limesurvey issues
- Last question of ranking. Selected option does not tally with output selection.
- Submission of survey was not recorded and brought forward to a Limesurvey page.

Issues
- No credits accounted after completion of survey
- After clicking submit, error message “/www/..” appeared before success page comes out.

User Experience
- Satisfaction Level – 83% were pleased with the app. Score of 7 and above.
- Easiness to use the app – 86% finds it easy to use the app. Score of 7 and above.
- Spelling error such as “OKK” button
- Slow response time upon clicks

Improvement(s):
- Question size could fit whole iOS screen.
- Quality control the survey question and template. If not, it will look back on the app. (Sageby, take note).
- Add more color to the survey page
- Can add a redemption button after completion of survey.
- No horizontal scrolling

3. Redemption Module

Easiness for redemption
- 84% gave a score of 6 and above for redeem a reward using the application

Distinguishable between store and wallet function
- 73% find it okay.

Receptiveness of QR Code
- 63% find it exciting

Crash Alert
- Reward > Wallet > Ok > Crashed

Issues
- No credits accounted after completion of survey


Improvement:
- Display cumulative credits earned at the top right hand corner at every page.
- Grey out area that can’t be redeem
- Instruction Tab about the app.
- Highlight how store and wallet are related
- Show redemption history


4. Application User Experience

Satisfaction Level – 83% were pleased with the app. Score of 7 and above.
Easiness to use the app – 86% finds it easy to use the app. Score of 7 and above.

Issues
- Spelling error such as “OKK” button
- Slow response time upon clicks


Main Changes to App


Tff.GIF


What can be improve


The app was too buggy for user test. There was confusion from bug crashes from login process to redemption process. Store feature was buggy and hence, nothing were shown in wallet. There were lack of quality control for Limesurvey template and questions.

Test users did not arrive as per scheduled which causes unforeseen bottleneck and also no show occurrences.


What we did well


There was a proper demonstration of process flow via 3 booths for different purposes. Proper instructions were given from the Testers to the users. All equipment’s are working fine. There were minimal waiting times.


Documentation

User Test 4 Review Minutes
User Test 4 Schedule
User Test 4 Test Plan
User Test 4 Summary