Difference between revisions of "IS480 Team wiki: 2016T2 LickiLicky User Test and Survey User Test 5"
(16 intermediate revisions by 3 users not shown) | |||
Line 95: | Line 95: | ||
Find out display preference among users | Find out display preference among users | ||
# Between these displays, which one do you prefer? (choose between A, B, and no preference) | # Between these displays, which one do you prefer? (choose between A, B, and no preference) | ||
+ | # How important is your current choice of display to you? (1 Not important, 5 very important) Skip if you have no preference | ||
<b> Action Table for Existing Functions </b> | <b> Action Table for Existing Functions </b> | ||
Line 161: | Line 162: | ||
<b> Action Table for Display Preference</b><br /> | <b> Action Table for Display Preference</b><br /> | ||
− | + | {| class="wikitable" style="text-align:justify; background: white;" | |
− | + | ! style="background: #FF80AB; color: white; font-weight: bold;" |Score | |
− | + | ! style="background: #FF80AB; color: white; font-weight: bold;" |Description | |
+ | ! style="background: #FF80AB; color: white; font-weight: bold;" |Action | ||
+ | |- | ||
+ | |rowspan="1"| preference >=70%<br />AND <br />mean rating =>3.0 | ||
+ | || UI change is in demand | ||
+ | || Implement UI change | ||
+ | |- | ||
+ | |rowspan="1"| preference <70%<br />OR <br />mean rating <3.0 | ||
+ | || No strong need for change | ||
+ | || No action | ||
+ | |- | ||
+ | |} | ||
<b>Recruitment: </b><br> | <b>Recruitment: </b><br> | ||
Line 182: | Line 194: | ||
## Planned improvements | ## Planned improvements | ||
## Planned functions | ## Planned functions | ||
− | # User is asked for their preference between Filter display in Figure 1 | + | # User is asked for their preference and rating for the choice between Filter display in Figure 1 |
# After the end of the user test. Actions are taken upon the suggestions and ratings based on the UT5 action table. | # After the end of the user test. Actions are taken upon the suggestions and ratings based on the UT5 action table. | ||
[[File:Lickilicky 0 Set A vs B.png | 800px]]<br /> | [[File:Lickilicky 0 Set A vs B.png | 800px]]<br /> | ||
− | |||
Figure 1<br /> | Figure 1<br /> | ||
− | + | '''UT 5 follow up for all participants'''<br /> | |
− | UT 5 follow up for all participants<br /> | + | # User is asked for their preference and rating for the choice between Filter display in Figure 1. Those who have answered in UT5 have the option to skip. |
− | + | # User is asked for their preference between Attach media displays (screenshots below) and provide a rating for their choice.<br /> | |
− | #User is asked for their preference between Attach media displays (screenshots below)<br /> | + | [[File:Lickilicky 1 note attach button.JPG | 530px]] |
− | |||
− | [[File:Lickilicky 1 note attach button.JPG | | ||
[[File:Lickilicky 1a note floating action button display choices.JPG | 800px]] | [[File:Lickilicky 1a note floating action button display choices.JPG | 800px]] | ||
<br> | <br> | ||
Line 210: | Line 219: | ||
<br>[[File:Ut5 note taking.png|800px]] | <br>[[File:Ut5 note taking.png|800px]] | ||
<br>[[File:Ut5 note retrieve.png|800px]] | <br>[[File:Ut5 note retrieve.png|800px]] | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
<br> | <br> | ||
<br><br> | <br><br> | ||
Line 225: | Line 226: | ||
<br>[[File:Ut5 reminder taking.png|800px]] | <br>[[File:Ut5 reminder taking.png|800px]] | ||
<br>[[File:Ut5 reminder retrieve.png|800px]] | <br>[[File:Ut5 reminder retrieve.png|800px]] | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
<br> | <br> | ||
<br><br> | <br><br> | ||
Line 239: | Line 233: | ||
<br> | <br> | ||
<br> <b> Insight </b> | <br> <b> Insight </b> | ||
+ | Some users prefer to have the "attach" icon to be on the top of the screen. | ||
<br> | <br> | ||
<br><br> | <br><br> | ||
Line 245: | Line 240: | ||
<br> We also surveyed our participants to validate the usefulness of our current functions. | <br> We also surveyed our participants to validate the usefulness of our current functions. | ||
<br> | <br> | ||
+ | [[File:UT5_Existing_Functions_Rating.png]] | ||
<br><br> | <br><br> | ||
Line 250: | Line 246: | ||
<br> We also surveyed our participants to validate the ideas of some of our possible future functions. | <br> We also surveyed our participants to validate the ideas of some of our possible future functions. | ||
<br> | <br> | ||
+ | [[File:Lickilicky UT5 New Features Rating.png]] | ||
<br><br> | <br><br> | ||
Line 255: | Line 252: | ||
<br> We also surveyed our participants to validate the ideas of some of our possible future improvements. | <br> We also surveyed our participants to validate the ideas of some of our possible future improvements. | ||
<br> | <br> | ||
+ | [[File:Lickilicky UT5 Improvement Rating.png]] | ||
<br><br> | <br><br> | ||
+ | |||
+ | <b>Goals & Data Revisited</b><br /> | ||
+ | |||
+ | Target users’ productivity is improved from using Tictacts <b>Passed.</b> | ||
+ | # Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process. <b>Passed.</b> | ||
+ | # Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process. <b>Passed.</b> | ||
+ | # Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process. <b>Passed.</b> | ||
+ | # Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process. <b>Passed.</b> | ||
+ | # Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above. <b>Passed,</b> | ||
+ | Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users <b>Passed,</b> with exception of Create Event 4/11 | ||
+ | # How will this feature impact your work process? Target: More than half of the participants provided positive feedback. | ||
+ | ## Notes 6/11 | ||
+ | ## Reminders 9/11 | ||
+ | ## Attach Media 8/11 | ||
+ | ## Create Event 4/11 (only Create Event did no meet the target) | ||
+ | ## Hashtag 8/11 | ||
+ | ## Tag Contact 8/11 | ||
+ | ## Filter 8/11 | ||
+ | ## Birthday Reminder 9/11 | ||
+ | # Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above. <b>Passed.</b> | ||
+ | ## All existing functions have a mean rating of 4 and above. | ||
+ | |||
+ | Find out improvements and features to the app | ||
+ | # Qualitative, question: Any suggestions to the interface or function? Rate the impact of the new functions (1 being not very useful - 7 being very useful) | ||
+ | ## [Birthday Reminder] Settings - on/off, add reminders n days before birthday. Suggested by P1 - 7, P2 - 7, P4 - 6, P5 -7, already in planned improvements. | ||
+ | ## [Birthday Reminder] Summarized birthday notification. Suggested by P5 - 6, already in planned improvements. | ||
+ | ## [Filter] Sort contacts by last added. Suggested by P1 - 6, added into UT5 planned improvements | ||
+ | ## [Contact, Birthday] Entering of birthday, birthday year, age along side with birthdate, P9 - 4, did not add to planned improvement due to low score. | ||
+ | # Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful) | ||
+ | ## SMS Appointment reminder. Suggested by P2 - 6, already in planned features. | ||
+ | |||
+ | Find out which planned improvements and features to the app are useful | ||
+ | # Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 5 and above. | ||
+ | ## '''Not so useful:''' [Reminder] Snooze for n mins, mean 4.94 | ||
+ | ## '''Not so useful:''' [Reminders] Optional time for reminder, mean 4.55 | ||
+ | ## '''Not so useful:''' [Birthday] Auto-sending of birthday SMS at 8am, mean 4.80 | ||
+ | ## '''Not so useful:''' [Birthday] Templates for birthdays text, mean 4.95 | ||
+ | ## '''Not so useful:''' [Filter] Sort contacts by last added, mean 4.57 | ||
+ | ## '''Very useful''' [Reminders] Deleting all done reminders at once, mean 6.00, implement in iter 15 as score is low. | ||
+ | # Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 5 and above. | ||
+ | ## All planned improvements mean score are above 5 | ||
+ | |||
+ | Find out display preference among users | ||
+ | # Between these displays, which one do you prefer? (choose between A, B, and no preference) | ||
+ | # How important is your current choice of display to you? (1 Not important, 5 very important) Skip if you have no preference | ||
+ | ## Filter display Preference | ||
+ | ### 65% prefers "Display B (New) - less fine, by alphabets" and the mean rating is 1.35 | ||
+ | ### 35% prefers "Display A (Original) - more fine, by the name of organization / title" and the mean rating is 1.29 | ||
+ | ### Summary: No strong need for change - No action | ||
+ | ## Attach button position in Edit Note preference | ||
+ | ### 71.4% prefers "Version B - Attach button at header" and mean rating is 3.07 | ||
+ | ### 19% prefers "Version A - Floating attach button at bottom right" and mean rating is 1.5 | ||
+ | ### 9.5% has no preference | ||
+ | ### Summary: '''UI change is in demand - Implement UI change''' | ||
+ | ## Attach buttons display in Notes List | ||
+ | ### 47.6% prefers "Version B - horizontal buttons" and mean rating is 1.9 | ||
+ | ### 33% prefers "Version A - vertical buttons" and mean rating is 1.57 | ||
+ | ### 19% prefers "Neither - I prefer to see the buttons after going into an individual note" and mean rating is 1.75 | ||
+ | ### 0% has "No preference" | ||
+ | ### Summary: No strong need for change - No action | ||
+ | ## Attach buttons display in Contact Notes List | ||
+ | ### 52.4% prefers "Version A - vertical buttons" and mean rating is 1.82 | ||
+ | ### 33% prefers "Version B - horizontal buttons" and mean rating is 1.86 | ||
+ | ### 14.3% prefers "Neither - I prefer to see the buttons after going into an individual note" and mean rating is 2 | ||
+ | ### 0% has "No preference" | ||
+ | ### Summary: No strong need for change - No action | ||
==Action Plan== | ==Action Plan== | ||
− | * | + | * Allow multiple / all reminders to be deleted at once. |
+ | * Attach button position in Edit Note preference - to adopt "Version B - Attach button at header" | ||
<!---<center><font size=3><b>Archived Project Info</b></font><br> | <!---<center><font size=3><b>Archived Project Info</b></font><br> |
Latest revision as of 14:15, 11 April 2016
HOME | ABOUT US | PROJECT OVERVIEW | PROJECT MANAGEMENT | DOCUMENTATION |
Technical Documents | Design Documents | User Test & Survey | Meeting Minutes | Course Deliverables |
---|
User Test 1 | User Test 2 | User Test 3 | User Test 4 | User Test 5 |
---|
Test Plan
Number of Users: 11
Date: 19 March 2016 - 28 March 2016
Venue: Anywhere convenient for the Financial Consultants and LickiLicky
Duration: 45-60mins
Objectives:
- Discover impact of existing functions, planned improvements & features
- Receive suggestions for improvements and new functions and gather feedback on them
- Determine if using Tictacts improves productivity by collecting specified task timings
Scope:
- Contacts & Notes
- Reminder & Notification
- Create Event
- Attach Media to Notes
- Hashtag & Mention
- Birthday Reminder
- Filter Search
Goals & Data
Target users’ productivity is improved from using Tictacts
- Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process.
- Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process.
- Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process.
- Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process.
- Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above.
Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users
- How will this feature impact your work process? Target: More than half of the participants provided positive feedback.
- Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above.
Find out which planned improvements and features to the app are useful
- Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 5 and above.
- Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 5 and above.
Find out improvements and features to the app
- Qualitative, question: Any suggestions to the interface or function?
- Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
Find out display preference among users
- Between these displays, which one do you prefer? (choose between A, B, and no preference)
- How important is your current choice of display to you? (1 Not important, 5 very important) Skip if you have no preference
Action Table for Existing Functions
Score | Description | Action |
---|---|---|
6.00 - 7.00 | Very useful | GOOD! 👍 |
5.00 - 5.99 | Useful | Ask for improvements to the existing function |
4.00 - 4.99 | Slightly useful | Ask for improvements to the existing function |
0.00 - 3.99 | Not useful | Feature is not useful, stop improvements and consider removing it feature list after FYP |
Action Table for Planned Functions & Improvements (Done After >= 5 Feedback Collected)
Score | Description | Action |
---|---|---|
6.00 - 7.00 | Very useful | Feature is useful, collect more ratings to rank. Add to tasks if effort is low. |
5.00 - 5.99 | Useful | Calculate score with the improved feature impact to determine whether to do it. Place the improved feature above the original feature for rating to avoid bias. |
0.00 - 4.99 | Not quite useful | Feature is not too useful and remove it from planned features list |
Action Table for New Functions & Improvements Suggested By Them
Score | Description | Action |
---|---|---|
6.00 - 7.00 | Very useful | Add to planned feature / improvements and collect more data |
5.00 - 5.99 | Useful | Review and include in planned feature/improvements if effort is below 3 |
0.00 - 4.99 | Slightly useful | Nevermind it |
Action Table for Display Preference
Score | Description | Action |
---|---|---|
preference >=70% AND mean rating =>3.0 |
UI change is in demand | Implement UI change |
preference <70% OR mean rating <3.0 |
No strong need for change | No action |
Recruitment:
Recruitment of Financial Consultants are done through peers and friends of these peers. In addition, recruitment is done via LinkedIn. A 60mins appointment is set up with them. Reimbursement through a S$10 Capital Mall voucher is made known to all potential participants to encourage participation.
Procedure:
- Users are first required to consent to participate in the user test.
- Demographics and job-related information are then collected
- An understanding of the user's current workflow is understood with a series of questions
- User is asked to perform the following tasks:
- Create new contact and note (timed)
- Create task/reminder & notification (timed)
- Attach media
- User is shown screenshots of other functions (Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter)
- Existing functions
- User is asked to provide suggestions on the interface and functions, providing a rating to the suggestion as well.
- User is finally asked to rate the
- Planned improvements
- Planned functions
- User is asked for their preference and rating for the choice between Filter display in Figure 1
- After the end of the user test. Actions are taken upon the suggestions and ratings based on the UT5 action table.
UT 5 follow up for all participants
- User is asked for their preference and rating for the choice between Filter display in Figure 1. Those who have answered in UT5 have the option to skip.
- User is asked for their preference between Attach media displays (screenshots below) and provide a rating for their choice.
Results & Analysis
Details of participants:
Total of 11 sign ups.
Findings:
Contacts & Notes
Comparison chart between the process of taking and retrieving of notes by using current process and using Tictacts
Reminders
Comparison chart between the process of creating and retrieving of reminders by using current process and using Tictacts
Attach Media to Notes
We find out how individual store a contact's details, that was recorded in hard copy. On the hard copy, the contact's birthday was also recorded.
Insight
Some users prefer to have the "attach" icon to be on the top of the screen.
Score of Current Functions
We also surveyed our participants to validate the usefulness of our current functions.
Score of Planned Functions
We also surveyed our participants to validate the ideas of some of our possible future functions.
Score of Planned Improvements
We also surveyed our participants to validate the ideas of some of our possible future improvements.
Goals & Data Revisited
Target users’ productivity is improved from using Tictacts Passed.
- Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process. Passed.
- Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process. Passed.
- Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process. Passed.
- Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process. Passed.
- Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above. Passed,
Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users Passed, with exception of Create Event 4/11
- How will this feature impact your work process? Target: More than half of the participants provided positive feedback.
- Notes 6/11
- Reminders 9/11
- Attach Media 8/11
- Create Event 4/11 (only Create Event did no meet the target)
- Hashtag 8/11
- Tag Contact 8/11
- Filter 8/11
- Birthday Reminder 9/11
- Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above. Passed.
- All existing functions have a mean rating of 4 and above.
Find out improvements and features to the app
- Qualitative, question: Any suggestions to the interface or function? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
- [Birthday Reminder] Settings - on/off, add reminders n days before birthday. Suggested by P1 - 7, P2 - 7, P4 - 6, P5 -7, already in planned improvements.
- [Birthday Reminder] Summarized birthday notification. Suggested by P5 - 6, already in planned improvements.
- [Filter] Sort contacts by last added. Suggested by P1 - 6, added into UT5 planned improvements
- [Contact, Birthday] Entering of birthday, birthday year, age along side with birthdate, P9 - 4, did not add to planned improvement due to low score.
- Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
- SMS Appointment reminder. Suggested by P2 - 6, already in planned features.
Find out which planned improvements and features to the app are useful
- Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 5 and above.
- Not so useful: [Reminder] Snooze for n mins, mean 4.94
- Not so useful: [Reminders] Optional time for reminder, mean 4.55
- Not so useful: [Birthday] Auto-sending of birthday SMS at 8am, mean 4.80
- Not so useful: [Birthday] Templates for birthdays text, mean 4.95
- Not so useful: [Filter] Sort contacts by last added, mean 4.57
- Very useful [Reminders] Deleting all done reminders at once, mean 6.00, implement in iter 15 as score is low.
- Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 5 and above.
- All planned improvements mean score are above 5
Find out display preference among users
- Between these displays, which one do you prefer? (choose between A, B, and no preference)
- How important is your current choice of display to you? (1 Not important, 5 very important) Skip if you have no preference
- Filter display Preference
- 65% prefers "Display B (New) - less fine, by alphabets" and the mean rating is 1.35
- 35% prefers "Display A (Original) - more fine, by the name of organization / title" and the mean rating is 1.29
- Summary: No strong need for change - No action
- Attach button position in Edit Note preference
- 71.4% prefers "Version B - Attach button at header" and mean rating is 3.07
- 19% prefers "Version A - Floating attach button at bottom right" and mean rating is 1.5
- 9.5% has no preference
- Summary: UI change is in demand - Implement UI change
- Attach buttons display in Notes List
- 47.6% prefers "Version B - horizontal buttons" and mean rating is 1.9
- 33% prefers "Version A - vertical buttons" and mean rating is 1.57
- 19% prefers "Neither - I prefer to see the buttons after going into an individual note" and mean rating is 1.75
- 0% has "No preference"
- Summary: No strong need for change - No action
- Attach buttons display in Contact Notes List
- 52.4% prefers "Version A - vertical buttons" and mean rating is 1.82
- 33% prefers "Version B - horizontal buttons" and mean rating is 1.86
- 14.3% prefers "Neither - I prefer to see the buttons after going into an individual note" and mean rating is 2
- 0% has "No preference"
- Summary: No strong need for change - No action
- Filter display Preference
Action Plan
- Allow multiple / all reminders to be deleted at once.
- Attach button position in Edit Note preference - to adopt "Version B - Attach button at header"