HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2016T2 LickiLicky User Test and Survey User Test 4"

From IS480
Jump to navigation Jump to search
(correct action table)
 
(14 intermediate revisions by 2 users not shown)
Line 75: Line 75:
 
* Birthday Reminder
 
* Birthday Reminder
 
* Filter Search
 
* Filter Search
 +
 +
<b>Goals & Data</b><br />
 +
 +
Target users’ productivity is improved from using Tictacts
 +
# Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process.
 +
# Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process.
 +
# Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process.
 +
# Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process.
 +
# Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above.
 +
Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users
 +
# How will this feature impact your work process? Target: More than half of the participants provided positive feedback.
 +
# Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above.
 +
Find out which planned improvements and features to the app are useful
 +
# Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
 +
# Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
 +
Find out improvements and features to the app
 +
# Qualitative, question: Any suggestions to the interface or function?
 +
# Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
 +
  
 
<b> Action Table for Existing Functions </b>
 
<b> Action Table for Existing Functions </b>
 +
{| class="wikitable" style="text-align:justify; background: white;"
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Score
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Description
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Action
 +
|-
 +
|rowspan="1"| 6.00 - 7.00
 +
|| Very useful
 +
|| GOOD! 👍
 +
|-
 +
|rowspan="1"| 4.00 - 5.99
 +
|| Slightly useful
 +
|| Ask for improvements to the existing function
 +
|-
 +
|rowspan="1"| 0.00 - 3.99
 +
|| Not useful
 +
|| Feature is not useful, stop improvements and consider removing it feature list after FYP
 +
|-
 +
|}
  
 
<b> Action Table for Planned Functions & Improvements </b>
 
<b> Action Table for Planned Functions & Improvements </b>
 +
{| class="wikitable" style="text-align:justify; background: white;"
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Score
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Description
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Action
 +
|-
 +
|rowspan="1"| 6.00 - 7.00
 +
|| Very useful
 +
|| Feature is useful and no action is needed
 +
|-
 +
|rowspan="1"| 4.00 - 5.99
 +
|| Slightly useful
 +
|| Can become useful if it is improved. Calculate score with the improved feature impact to determine whether to do it. <br>Place the improved feature above the original feature for rating to avoid bias. Cannot be improved. Get more data.
 +
|-
 +
|rowspan="1"| 0.00 - 3.99
 +
|| Not useful
 +
|| Feature is not useful and remove it from planned features list
 +
|-
 +
|}
  
 
<b> Action Table for New Functions & Improvements</b>
 
<b> Action Table for New Functions & Improvements</b>
 +
{| class="wikitable" style="text-align:justify; background: white;"
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Score
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Description
 +
! style="background: #FF80AB; color: white; font-weight: bold;" |Action
 +
|-
 +
|rowspan="1"| 6.00 - 7.00
 +
|| Very useful
 +
|| Add to plan feature list and collect more data
 +
|-
 +
|rowspan="1"| 4.00 - 5.99
 +
|| Slightly useful
 +
|| Ask why they suggested it then. Review and include if effort is low and passes the feature threshold
 +
|-
 +
|rowspan="1"| 0.00 - 3.99
 +
|| Not useful
 +
|| If it is a interface or usability issue, then backlog it.If feature, then nevermind it.
 +
|-
 +
|}
 +
 +
<b>Recruitment: </b><br>
 +
Recruitment of Financial Consultants are done through peers and friends of these peers. A 60mins appointment is set up with them. Reimbursement through a S$10 Capital Mall voucher is made known to all potential participants to encourage participation.
  
<b>Procedure: </b>xxx <br>
+
<b>Procedure: </b><br>
 +
# Users are first required to consent to participate in the user test.
 +
# Demographics and job-related information are then collected
 +
# An understanding of the user's current workflow is understood with a series of questions
 +
# User is asked to perform the following tasks:
 +
## Create new contact and note (timed)
 +
## Create task/reminder & notification (timed)
 +
## Create event
 +
## Attach media
 +
## Hashtag & Mention
 +
## Birthday reminder
 +
## Filter contacts
 +
# User is asked to rate the
 +
## Existing functions
 +
# User is asked to provide suggestions on the interface and functions, providing a rating to the suggestion as well.
 +
# User is finally asked to rate the
 +
## Planned improvements
 +
## Planned functions
 +
# After the end of the user test. Actions are taken upon the suggestions and ratings based on the UT4 action table.
  
 
==Results & Analysis==
 
==Results & Analysis==
 
<b>Details of participants:</b>
 
<b>Details of participants:</b>
 
<br> Total of 11 sign ups.
 
<br> Total of 11 sign ups.
<br>[[File:|400px]]
+
<br>[[File:Ut4 companies.png|800px]]
[[File:|400px]]
+
<br> <br>
  
 
<b>Findings:</b>
 
<b>Findings:</b>
 
<b> Contacts & Notes </b>
 
<b> Contacts & Notes </b>
<br> First, we find out about their as-is process of taking and retrieving notes
+
<br> Comparison chart between the process of taking and retrieving of notes by using current process and using Tictacts
<br>[[File:Note Taking As-Is.png|400px]]
+
<br>[[File:Ut4 note taking.png|800px]]
[[File:|400px]]]
+
<br>[[File:Ut4 note retrieve.png|800px]]]
 +
<br>
 +
<br><b> Insight </b>
 +
<br>[[File:Ut4 note insight.png|800px]]
 +
<br> <br> Current Process:
 +
<ul>
 +
<li>CRM: Need to re-run program if password is wrong
 +
<li>Hardcopy: Messy, disorganized
 +
</ul>
 +
<br> Why not e-form:
 +
<ul>
 +
<li>Comfortable in using paper survey form
 +
<li>E-form may be too structure, paper offers flexibility
 +
<li>Easier to use paper to pitch, more professional
 +
</ul>
 +
 
 +
<br>Comments:
 +
<br>10/11 commented that notes function helps improve their work. It is a single access point for individual contact. However, more effort has to be invested in creating notes.
 
<br><br>
 
<br><br>
Next, we introduce Tictacts and test how will participants approach the same scenario using the app.
 
<br><br>[[File:|600px]]
 
  
 
<b> Reminders </b>
 
<b> Reminders </b>
<br> We find out about their as-is process of keeping and retrieving reminders
+
<br> Comparison chart between the process of creating and retrieving of reminders by using current process and using Tictacts
<br>[[File:Note Taking As-Is.png|400px]]
+
<br>[[File:Ut4 reminder taking.png|800px]]
[[File:|400px]]]
+
<br>[[File:Ut4 reminder retrieve.png|800px]]]
 +
<br>
 +
<br><b> Insight </b>
 +
<br>[[File:Ut4 reminder insight.png|800px]]
 +
<br> <br> Current Process:
 +
<br>Hardcopy & Excel: May forget about task. Need manual reference. Memory based approached.
 +
<br>
 +
<br> Comments:
 +
<br>7/11 commented that reminder function is useful to them and the option to call during notification is good.
 
<br><br>
 
<br><br>
Next, we introduce Tictacts and test how will participants approach the same scenario using the app.
 
<br><br>[[File:|600px]]
 
  
 
<b> Create Event </b>
 
<b> Create Event </b>
<br> We find out how individual remember and note down future event with another person.
+
<br>[[File:Ut4 create event.png|800px]]
<br>[[File:|400px]]
+
<br> <b> Insight </b>
 +
<br>Financial Consultants mentioned they prefer to stick to current process. Also, they added that sending text to attendees may be better as email is not the primary mode to contact client.
 
<br><br>
 
<br><br>
xxx
 
<br>[[File:|600px]]
 
  
 
<b> Attach Media to Notes </b>
 
<b> Attach Media to Notes </b>
 
<br> We find out how individual store a contact's details, that was recorded in hard copy. On the hard copy, the contact's birthday was also recorded.
 
<br> We find out how individual store a contact's details, that was recorded in hard copy. On the hard copy, the contact's birthday was also recorded.
<br>[[File:Attach Media to Notes As Is.png|400px]]
+
<br>[[File:Ut4 hardcopy ways.png|800px]]
[[File:|400px]]
+
<br> <b> Insight </b>
 +
<br>Financial consultants mentioned that they seldom retrieve such information. However, problem may arise as current process is not organized.
 +
<br><br>
 +
 
 +
<b> Score of Current Functions </b>
 +
<br> We also surveyed our participants to validate the usefulness of our current functions.
 +
<br>[[File:Ut4 current feature.png|800px]]
 +
<br><br>
 +
 
 +
<b> Score of Planned Functions </b>
 +
<br> We also surveyed our participants to validate the ideas of some of our possible future functions.
 +
<br>[[File:Lickilicky UT4 New Features Rating.png]]
 
<br><br>
 
<br><br>
We observed that only 13 participants took a picture of the hard copy details and attached it to the note for the contact. 15 feedback that they are unsure of what does the "attach" icon represent. For birthday reminder, 4 mentioned that they prefer to type in the dates manually and another 4 mentioned that the birthday reminder idea is useful.
 
<br>[[File:|600px]]
 
  
<b> Hashtag & Mention</b>  
+
<b> Score of Planned Improvements </b>
 +
<br> We also surveyed our participants to validate the ideas of some of our possible future improvements.
 +
<br>[[File:Lickilicky UT4 Improvement Rating.png]]
 +
<br><br>
  
<b> Birthday Reminder</b>  
+
<b>Goals & Data Revisited</b><br />
  
<b> Filter Search </b>
+
Target users’ productivity is improved from using Tictacts <b>Passed.</b>
<br> We also find out how individual locate the hard copy details, based on the method which they mentioned previous. The following is a summary of our findings
+
# Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process. <b>Passed.</b>
<br>[[File:|600px]]
+
# Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process. <b>Passed.</b>
 +
# Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process. <b>Passed.</b>
 +
# Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process. <b>Passed.</b>
 +
# Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above. <b>Passed.</b>
 +
Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users <b>Passed,</b> with the exception of Create Event which only have 5/11 positive feedback.
 +
# How will this feature impact your work process? Target: More than half of the participants provided positive feedback.
 +
## Notes 10/11
 +
## Reminders 7/11
 +
## Create Event 5/11 (only Create Event did not meet the target)
 +
## Attach Media 8/11
 +
## Hashtag  9/11
 +
## Tag Contact 7/11
 +
## Birthday Reminder 9/11
 +
## Filter 7/11
 +
# Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above. <b>Passed.</b>
 +
## All existing functions have a mean rating of 4 and above.
  
<b> </b>
+
Find out improvements and features to the app
 +
# Qualitative, question: Any suggestions to the interface or function? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
 +
## [Filter] Result list after filtering by 'Organization' is too fine. putting A-Z will be okay. P4 - 6 . Added to questionnaire for UT5
 +
## [Birthday] Entering of birthday, birthday year, age along side with birthdate, P2 - 5. Added to planned improvements for UT4.
 +
## [Reminders] Snooze for n mins. P1 - 6, P8 - 7 Added to planned improvements for UT4.
 +
## [Reminder] Refine headers for upcoming reminders (tdy, tmr, next 7 days, later). P2 - 6. Added to planned improvements for UT4.
 +
## [Reminder] Show pop up when screen is locked. P6 - 7. Added to planned improvements for UT4.
 +
## [Reminders] See reminders for time period P4 - 7 Added to planned improvements for UT5.
 +
## [Reminders] Deleting all done reminders & notes at once P4 - 7 Added to planned improvements for UT5.
 +
## [Reminders] Create reminder w/o alert (no time then dont alert).  P4 - 5 Added to planned improvements for UT5.
 +
## [Birthday Reminder] Summarized birthday notification P10 - 6.  Added to planned improvements for UT5.
 +
## [Birthday Reminder] Settings - on/off, add reminders n days before birthday P2 - 5, P6 - 7.  Added to planned improvements for UT5.
  
<b> Ranking of Current Functions </b>
+
# Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
<br> Towards the end of UT3, we asked our participants to rank the importance the functions which they had tested.
+
## Keep in touch reminders (Impact score 7) was suggested by P1 early and added to UT4 planned features.
<br>[[File:|600px]]
+
## SMS reminder for appointment was suggested by P9 - 7, and 5 other participants in remarks without impact score. Added to UT5 planned features.
 +
## Backup (already in planned features) was suggested by P6 with impact score of 7.
 +
## Calendar & events (already in planned features) was suggested by P6 - 7, P10 - 7.
  
<b> Ranking of Current Functions </b>
+
Find out which planned improvements and features to the app are useful
<br> We also surveyed our participants to validate the ideas of some of our possible future functions.
+
# Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
<br>[[File:|600px]]
+
## '''Not useful:''' Visualization of contact interactions is removed before the start of UT5 as the impact score 4.63 is below the threshold of 5.
 +
# Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
 +
## '''Not useful:''' [Contact] Filter > Sort contacts by most/least contacted. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.78
 +
## '''Not useful:''' [Contact] Short list of recently contacted contacts above regular A-Z contacts. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.88
 +
## '''Not useful:''' [Contact, Birthday] Entering of birthday, birthday year, age along side with birthdate. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.33
 +
## '''Not useful:''' [Contact] Differentiate whatsapp and regular phone number in Contact View. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.50
 +
## '''Useful:''' [Reminder] Refine headers for upcoming reminders (tdy, tmr, next 7 days, later). mean 6 mode 6 median 6.00. Implemented in Iter 15.
 +
## '''Useful:'''[Reminder] Show pop up when screen is locked. mean 7 mode 7 median 5.67. Implemented in Iter 14.
  
 
==Action Plan==
 
==Action Plan==
* Evaluate feedback and prioritise functional and usability issues to resolve.
+
* Building on improvement for reminder in Iteration 15 : Refine headers for upcoming reminders (median 6, mode 6, mean 6)
* Improve on reliability and user experience
+
* Building on improvement for reminder in Iteration 14 : Show pop up when screen is locked (median 7, mode 7)
* Analyse which future function is most preferred by testers and determine which feature to be added / remove from timeline
 
* Identify participants work processes towards the scenarios given and analyse if current product will improve productivity
 
  
 
<!---<center><font size=3><b>Archived Project Info</b></font><br>
 
<!---<center><font size=3><b>Archived Project Info</b></font><br>

Latest revision as of 18:08, 18 April 2016

LickiLicky Logo v2.png

HOME   ABOUT US   PROJECT OVERVIEW   PROJECT MANAGEMENT   DOCUMENTATION
Technical Documents Design Documents User Test & Survey Meeting Minutes Course Deliverables


User Test & Survey
User Test 1 User Test 2 User Test 3 User Test 4 User Test 5
User Test 4

Test Plan

Number of Users: 11
Date: 7 March 2016 - 16 March 2016
Venue: Anywhere convenient for the Financial Consultants and LickiLicky
Duration: 45-90mins
Objectives:

  • Discover impact of existing functions, planned improvements & features
  • Receive suggestions for improvements and new functions and gather feedback on them
  • Determine if using Tictacts improves productivity by collecting specified task timings

Scope:

  • Contacts & Notes
  • Reminder & Notification
  • Create Event
  • Attach Media to Notes
  • Hashtag & Mention
  • Birthday Reminder
  • Filter Search

Goals & Data

Target users’ productivity is improved from using Tictacts

  1. Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process.
  2. Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process.
  3. Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process.
  4. Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process.
  5. Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above.

Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users

  1. How will this feature impact your work process? Target: More than half of the participants provided positive feedback.
  2. Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above.

Find out which planned improvements and features to the app are useful

  1. Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
  2. Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.

Find out improvements and features to the app

  1. Qualitative, question: Any suggestions to the interface or function?
  2. Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful)


Action Table for Existing Functions

Score Description Action
6.00 - 7.00 Very useful GOOD! 👍
4.00 - 5.99 Slightly useful Ask for improvements to the existing function
0.00 - 3.99 Not useful Feature is not useful, stop improvements and consider removing it feature list after FYP

Action Table for Planned Functions & Improvements

Score Description Action
6.00 - 7.00 Very useful Feature is useful and no action is needed
4.00 - 5.99 Slightly useful Can become useful if it is improved. Calculate score with the improved feature impact to determine whether to do it.
Place the improved feature above the original feature for rating to avoid bias. Cannot be improved. Get more data.
0.00 - 3.99 Not useful Feature is not useful and remove it from planned features list

Action Table for New Functions & Improvements

Score Description Action
6.00 - 7.00 Very useful Add to plan feature list and collect more data
4.00 - 5.99 Slightly useful Ask why they suggested it then. Review and include if effort is low and passes the feature threshold
0.00 - 3.99 Not useful If it is a interface or usability issue, then backlog it.If feature, then nevermind it.

Recruitment:
Recruitment of Financial Consultants are done through peers and friends of these peers. A 60mins appointment is set up with them. Reimbursement through a S$10 Capital Mall voucher is made known to all potential participants to encourage participation.

Procedure:

  1. Users are first required to consent to participate in the user test.
  2. Demographics and job-related information are then collected
  3. An understanding of the user's current workflow is understood with a series of questions
  4. User is asked to perform the following tasks:
    1. Create new contact and note (timed)
    2. Create task/reminder & notification (timed)
    3. Create event
    4. Attach media
    5. Hashtag & Mention
    6. Birthday reminder
    7. Filter contacts
  5. User is asked to rate the
    1. Existing functions
  6. User is asked to provide suggestions on the interface and functions, providing a rating to the suggestion as well.
  7. User is finally asked to rate the
    1. Planned improvements
    2. Planned functions
  8. After the end of the user test. Actions are taken upon the suggestions and ratings based on the UT4 action table.

Results & Analysis

Details of participants:
Total of 11 sign ups.
Ut4 companies.png

Findings: Contacts & Notes
Comparison chart between the process of taking and retrieving of notes by using current process and using Tictacts
Ut4 note taking.png
Ut4 note retrieve.png]

Insight
Ut4 note insight.png

Current Process:

  • CRM: Need to re-run program if password is wrong
  • Hardcopy: Messy, disorganized


Why not e-form:

  • Comfortable in using paper survey form
  • E-form may be too structure, paper offers flexibility
  • Easier to use paper to pitch, more professional


Comments:
10/11 commented that notes function helps improve their work. It is a single access point for individual contact. However, more effort has to be invested in creating notes.

Reminders
Comparison chart between the process of creating and retrieving of reminders by using current process and using Tictacts
Ut4 reminder taking.png
Ut4 reminder retrieve.png]

Insight
Ut4 reminder insight.png

Current Process:
Hardcopy & Excel: May forget about task. Need manual reference. Memory based approached.

Comments:
7/11 commented that reminder function is useful to them and the option to call during notification is good.

Create Event
Ut4 create event.png
Insight
Financial Consultants mentioned they prefer to stick to current process. Also, they added that sending text to attendees may be better as email is not the primary mode to contact client.

Attach Media to Notes
We find out how individual store a contact's details, that was recorded in hard copy. On the hard copy, the contact's birthday was also recorded.
Ut4 hardcopy ways.png
Insight
Financial consultants mentioned that they seldom retrieve such information. However, problem may arise as current process is not organized.

Score of Current Functions
We also surveyed our participants to validate the usefulness of our current functions.
Ut4 current feature.png

Score of Planned Functions
We also surveyed our participants to validate the ideas of some of our possible future functions.
Lickilicky UT4 New Features Rating.png

Score of Planned Improvements
We also surveyed our participants to validate the ideas of some of our possible future improvements.
Lickilicky UT4 Improvement Rating.png

Goals & Data Revisited

Target users’ productivity is improved from using Tictacts Passed.

  1. Quantitative, question / observation: How much time does it take for you to enter the contact and note into your current work process database? Target: Mean time taken for Tictacts Task 1 Keep is lower than mean time taken for their As-Is process. Passed.
  2. Quantitative, question / observation: How much time does it take for you to retrieve the contact and note from your database? Target: Mean time taken for Tictacts Task 1 Retrieve is lower than mean time taken for their As-Is process. Passed.
  3. Quantitative, question / observation: How much time does it take for you to keep the reminder into your database? Target: Mean time taken for Tictacts Task 2 Keep is lower than mean time taken for their As-Is process. Passed.
  4. Quantitative, question / observation: How much time does it take for you to retrieve the reminder from your database? Target: Mean time taken for Tictacts Task 2 Retrieve is lower than mean time taken for their As-Is process. Passed.
  5. Qualitative, question: Please rate the impact of Tictacts on your productivity. (Negative 1 - Neutral 3 - Positive 5) Target: Mean rating is 4 and above. Passed.

Existing functions (Notes, Reminders, Attach Media, Create Event, Hashtag, Tag Contact, Birthday Reminder, Filter) are useful to our target users Passed, with the exception of Create Event which only have 5/11 positive feedback.

  1. How will this feature impact your work process? Target: More than half of the participants provided positive feedback.
    1. Notes 10/11
    2. Reminders 7/11
    3. Create Event 5/11 (only Create Event did not meet the target)
    4. Attach Media 8/11
    5. Hashtag 9/11
    6. Tag Contact 7/11
    7. Birthday Reminder 9/11
    8. Filter 7/11
  2. Quantitative, question: Please rate functions in terms of their positive impact on your work process (1 Not useful - 7 Very useful) Target: Mean rating is 4 and above. Passed.
    1. All existing functions have a mean rating of 4 and above.

Find out improvements and features to the app

  1. Qualitative, question: Any suggestions to the interface or function? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
    1. [Filter] Result list after filtering by 'Organization' is too fine. putting A-Z will be okay. P4 - 6 . Added to questionnaire for UT5
    2. [Birthday] Entering of birthday, birthday year, age along side with birthdate, P2 - 5. Added to planned improvements for UT4.
    3. [Reminders] Snooze for n mins. P1 - 6, P8 - 7 Added to planned improvements for UT4.
    4. [Reminder] Refine headers for upcoming reminders (tdy, tmr, next 7 days, later). P2 - 6. Added to planned improvements for UT4.
    5. [Reminder] Show pop up when screen is locked. P6 - 7. Added to planned improvements for UT4.
    6. [Reminders] See reminders for time period P4 - 7 Added to planned improvements for UT5.
    7. [Reminders] Deleting all done reminders & notes at once P4 - 7 Added to planned improvements for UT5.
    8. [Reminders] Create reminder w/o alert (no time then dont alert). P4 - 5 Added to planned improvements for UT5.
    9. [Birthday Reminder] Summarized birthday notification P10 - 6. Added to planned improvements for UT5.
    10. [Birthday Reminder] Settings - on/off, add reminders n days before birthday P2 - 5, P6 - 7. Added to planned improvements for UT5.
  1. Quantitative, question: Any new functions and features you'd like to see? Rate the impact of the new functions (1 being not very useful - 7 being very useful)
    1. Keep in touch reminders (Impact score 7) was suggested by P1 early and added to UT4 planned features.
    2. SMS reminder for appointment was suggested by P9 - 7, and 5 other participants in remarks without impact score. Added to UT5 planned features.
    3. Backup (already in planned features) was suggested by P6 with impact score of 7.
    4. Calendar & events (already in planned features) was suggested by P6 - 7, P10 - 7.

Find out which planned improvements and features to the app are useful

  1. Quantitative, question: For the following planned improvements, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
    1. Not useful: Visualization of contact interactions is removed before the start of UT5 as the impact score 4.63 is below the threshold of 5.
  2. Quantitative, question: For the following planned features, please rate how much positive impact they will have. 1 being Very unimportant, and 7 being Very important. Target: Mean rating is 4 and above.
    1. Not useful: [Contact] Filter > Sort contacts by most/least contacted. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.78
    2. Not useful: [Contact] Short list of recently contacted contacts above regular A-Z contacts. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.88
    3. Not useful: [Contact, Birthday] Entering of birthday, birthday year, age along side with birthdate. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.33
    4. Not useful: [Contact] Differentiate whatsapp and regular phone number in Contact View. Removed after UT4 as action table 'keep score' is raised to 5.0, and impact score was 4.50
    5. Useful: [Reminder] Refine headers for upcoming reminders (tdy, tmr, next 7 days, later). mean 6 mode 6 median 6.00. Implemented in Iter 15.
    6. Useful:[Reminder] Show pop up when screen is locked. mean 7 mode 7 median 5.67. Implemented in Iter 14.

Action Plan

  • Building on improvement for reminder in Iteration 15 : Refine headers for upcoming reminders (median 6, mode 6, mean 6)
  • Building on improvement for reminder in Iteration 14 : Show pop up when screen is locked (median 7, mode 7)