Difference between revisions of "IS480 Team wiki: 2015T1 4Sight User Testing 2"
(31 intermediate revisions by the same user not shown) | |||
Line 98: | Line 98: | ||
====Number of Clicks==== | ====Number of Clicks==== | ||
+ | <br/> | ||
[[File:4Sight UT2 Claim1NoOfClicks.png|600px|center|link=]]<br/> | [[File:4Sight UT2 Claim1NoOfClicks.png|600px|center|link=]]<br/> | ||
− | Based on the above chart, create appointment results has the largest deviation compared to the other features. Many users encountered the same issue - activation of heat map does not navigate the calendar to the current date. This has caused many confusions and additional clicks to toggle back to the current date heatmap view if they are on a different week view. Based on these feedback, we have made some changes to improve users' experience in using the heat map by removing the unnecessary clicks (toggle to the current date). In the improved version, user will be brought to the current week's heatmap upon selecting "create new appointment" form | + | Based on the above chart, create appointment results has the largest deviation compared to the other features. Many users encountered the same issue - activation of heat map does not navigate the calendar to the current date. This has caused many confusions and additional clicks to toggle back to the current date heatmap view if they are on a different week view. Based on these feedback, we have made some changes to improve users' experience in using the heat map by removing the unnecessary clicks (toggle to the current date). In the improved version, user will be brought to the current week's heatmap upon selecting "create new appointment" form. |
− | [[File:4Sight UT2 claim1Changes.png|900px|center|link=]]<br/ | + | <br/> |
+ | [[File:4Sight UT2 claim1Changes.png|900px|center|link=]]<br/> | ||
====Number of Errors==== | ====Number of Errors==== | ||
[[File:4Sight UT2 Claim1NoOfErrors.png|600px|center|link=]]<br/> | [[File:4Sight UT2 Claim1NoOfErrors.png|600px|center|link=]]<br/> | ||
− | Based on the above chart, create and update appointment have the highest number of errors, 3 and 2 respectively. Many users encountered the same issue - Participants change dropdown selections before heat map finishes loading. | + | Based on the above chart, create and update appointment have the highest number of errors, 3 and 2 respectively. Many users encountered the same issue - Participants change dropdown selections before heat map finishes loading. Based on the feedback we have gotten, we have made improvement to prevent users from selecting the options in the dropdown too quickly. |
[[File:4Sight UT2 claim2Changes.png|900px|center|link=]]<br/> | [[File:4Sight UT2 claim2Changes.png|900px|center|link=]]<br/> | ||
− | === | + | ====Claim 1 Results==== |
+ | <br/> | ||
+ | [[File:4Sight UT2 claim1Score.png|800px|center|link=]]<br/> | ||
− | < | + | Taking the average of the component scores, the average mean score is 4.39 > 4. This supports our claim that participants find it easy to navigate the calendar and manage patients' appointments (CUD) effectively. |
− | + | ||
− | + | ||
− | + | <b>Some of the feedback/ comments made by the users are as follow:</b> | |
− | + | * Interface is user-friendly, but one small thing is it’s a bit of a hassle to toggle between Dr. Ho and Dr Goh at some points of time | |
− | + | * Flow of entering information is natural and does not feel awkward. However, there must be a choice to save appointment without selecting mode of referral | |
− | + | * It is easy to identify the appointment types with the help of color coding | |
− | + | * I like how easy it is to switch between doctor's calendar and viewing the appointment types on the calendar | |
− | + | ||
+ | <br/> | ||
+ | ===Claim 2 Results=== | ||
+ | <b>Claim:</b> Participants find the patient queue useful in helping them to track patients’ attendance (average rating: 4/5) | ||
+ | |||
+ | [[File:4Sight UT2 Claim2NoOfClicks.png|700px|center|link=]]<br/> | ||
+ | |||
+ | The number of clicks users took to perform the tasks are consistent with the average number of clicks by the members. This shows that the user interface is intuitive and easy to use. There is also no mis-interpretation of the various features in the queue page. Generally all the users found it easy to manage patient queue. | ||
+ | |||
+ | [[File:4Sight UT2 Claim2NoOfErrors.png|700px|center|link=]]<br/> | ||
+ | |||
+ | There are no errors in most of the tasks in the queue management module. The only error found was that the user could not associate the cross button to add to the no show list initially. | ||
+ | |||
+ | [[File:4Sight UT2 claim2Score.png|700px|center|link=]]<br/> | ||
+ | |||
+ | Overall the feedback gotten from this module was positive. The average score across all the tasks was 4.47 > 4, supporting our claim of participants finding the patient queue useful in helping them to track patients' attendance. | ||
+ | |||
+ | <b>Below are some of the feedback from users:</b> | ||
+ | * No show is good for keeping track of patient | ||
+ | * The patient queue page is very easy to use | ||
+ | * The no show feature helps save a lot of manual work required and time spent on compiling a list of no show list. The current work process requires us to manually check the appointments in the black book against the CMS and marketing form google spreadsheet that is managed by the marketing team. This is highly tedious and error prone. | ||
+ | |||
+ | ===Claim 3 Results=== | ||
+ | <b>Claim:</b> Participants find the marketing conversion and appointment scheduling dashboard beneficial in their performance analysis | ||
− | + | [[File:4Sight UT2 claim3Score.png|700px|center|link=]]<br/> | |
− | |||
− | |||
− | | | ||
− | |||
− | |||
− | |||
− | + | For this claim, we are interested in finding out on the usability of the analytics module. Taking the average of the scores across different tasks, we have gotten an average of 3.975 < 4. This is slightly less than what we have claimed. Some of the reasons we have gathered that explained the scores given are as follow: | |
− | + | * Include feature for exporting to excel (table) | |
− | + | * Analytics dashboards are useful but some of the charts are quite difficult to interpret | |
− | |||
− | |||
− | |||
− | |||
− | + | ===Claim 4 Results=== | |
− | + | <b>Claim:</b> Participants like and have positive comments about our user interface (average rating: 4/5) | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | </ | ||
+ | [[File:4Sight UT2 claim4Score.png|700px|center|link=]]<br/> | ||
− | + | In general, users like our user interface and have positive feedback about it. | |
− | |||
− | |||
− | |||
+ | ===Collated Feedback Score - Survey Questionnaire=== | ||
<div style="text-align: left; width:99%; margin-left:auto; margin-right:auto;"> | <div style="text-align: left; width:99%; margin-left:auto; margin-right:auto;"> | ||
{| class="wikitable" width="100%" font-family="Roboto Lt" | {| class="wikitable" width="100%" font-family="Roboto Lt" | ||
|- | |- | ||
− | ! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width: | + | ! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:5%; border:1px solid #999" | # |
− | ! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width: | + | ! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:30%; border:1px solid #999" | Question |
− | ! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width: | + | ! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:60%; border:1px solid #999" | Collated Comments |
− | |||
|- | |- | ||
| style="text-align:center" | 1 | | style="text-align:center" | 1 | ||
− | | style="text-align:center" | | + | | style="text-align:center" | What do you like about the application? |
− | | style="text-align:center" | | + | | style="text-align:center" | Very simple to use, simple to understand <br> Will make us efficient <br> Ease of use for essential tasks. Runs smoothly with no lag <br> I like the analytics dashboards as it simplifies the current process where a lot of my work was done manually, across multiple channels (CMS, paper, online shared doc and black book). An integrated platform like this helps reduce my workload <br> The app is easy to use. It will save us a lot of time especially in managing the appointments <br> Very simple to use and many of the features built are what we need <br> Very clear and nice <br> Very easy to use. Reduces human error, making us more efficient |
|- | |- | ||
| style="text-align:center" | 2 | | style="text-align:center" | 2 | ||
− | | style="text-align:center" | | + | | style="text-align:center" | What do you not like about the application? |
− | | style="text-align:center" | | + | | style="text-align:center" | Disable auto swap? Maybe can prompt to swap appointment when you delete appointment on crowded days <br> Learning curve for swap waiting list patients is steep <br> Auto sending out of appointment reminder. Some patients might not like it <br> Quite hard to understand and use the swap appointment function |
|- | |- | ||
| style="text-align:center" | 3 | | style="text-align:center" | 3 | ||
− | | style="text-align:center" | | + | | style="text-align:center" | Comment on how easy it is to use the application |
− | | style="text-align:center" | | + | | style="text-align:center" | Easy to navigate around and update appointments <br> Application is intuitive <br> very easy to navigate |
|- | |- | ||
| style="text-align:center" | 4 | | style="text-align:center" | 4 | ||
− | | style="text-align:center" | | + | | style="text-align:center" | Comment on the overall usefulness of the application in helping you complete your daily tasks? |
− | | style="text-align:center" | | + | | style="text-align:center" | Compared to the current work process, this application greatly reduces the time needed for us to complete some tasks such as calling patients to remind them of appointment, collating the patient no show list etc. <br> Eliminates many of the manual work that we have been doing. Save us a lot of time and reduces error especially during peak periods. <br> Maintains old function of our appointments book, and can be updated <br> I like the analytics dashboards as it simplifies the current process where a lot of my work was done manually, across multiple channels (CMS, paper, online shared doc and black book). An integrated platform like this helps reduce my workload <br> Streamlined work process <br> Charts provided are useful as it allows drilling down of information <br> Application reflects current functions but make a lot easier |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
|} | |} | ||
</div> | </div> | ||
− | |||
===User Study Summary=== | ===User Study Summary=== | ||
− | * Overall, users find the application easy to user | + | * Overall, users find the application easy to user and intuitive to perform all required tasks |
* Usability scores did not vary drastically from user to user | * Usability scores did not vary drastically from user to user | ||
− | * | + | * Number of clicks are fairly consistent compared to the members |
− | * User feedback for | + | * Users are satisfied and have positive comments on all the features of the application |
+ | * User provided good feedback for improvement | ||
===User Study Takeaways=== | ===User Study Takeaways=== | ||
− | * | + | * Consistent usability score for different users, considering that different users have different functions to perform in their day-to-day operation. |
− | * With users' feedback, we identified areas of improvement on these functions. Eg. | + | * With users' feedback, we identified areas of improvement on these functions. Eg. export function. |
− | * Allow us to evaluate the usefulness of these features. | + | * Allow us to evaluate the usefulness of these features. Team will improve on the features based on users feedback and manage all change requests (remove/ add feature). Most feedback has been documented and accounted for in our implementation plan in which the team will work on it in subsequent development. |
Latest revision as of 09:33, 16 November 2015
HOME | ABOUT US | PROJECT OVERVIEW | PROJECT MANAGEMENT | DOCUMENTATION |
Technical Diagrams | Design Documents | Meeting Minutes | User Testings | Project Documents |
Venue
- Location: Clearvision @ 6 Nutmeg Road
- Date: 16 October 2015 (Wednesday)
- Time: 15:00pm
Objectives
- Verify that functionalities built are in line with user requirements
- Determine if the user interface is intuitive
- Gather feedback on current functionalities
- Identify usability problems
Claims
Our team made 4 claims -
1. Participants find it easy to navigate the calendar and manage patients’ appointments (CUD) effectively (average rating: 4/5)
2. Participants find the patient queue useful in helping them track patient’s attendance (average rating: 3.5/5)
3. Participants find the marketing conversion and appointment scheduling dashboard beneficial in their performance analysis (average rating: 4/5)
4. Participants like and have positive comments about our user interface (average rating: 4/5)
Participants
- Number of participants: 8
- Roles of participants: Optometrist, admin clerk and marketing team member
Tasks
Each of the users was given a list of tasks to complete. Click here to download tasklist. The tasklist also consists of some questionnaire to gather feedback from users.
Summary of Survey Results
Click here to download the raw data.
Usability Results
Usability Analysis
- No drastic variation in the usability scores between users of different roles
- Simple UI that allows user to perform fundamental tasks easily
- Confirmed the value of proposed features such as swap list
- Discover potential improvement eg: Block off doctor’s time slots on calendar
Claim 1 Results
Claim: Participants find it easy to navigate the calendar and manage patients' appointments (CUD) effectively (average rating: 4/5)
Number of Clicks
Based on the above chart, create appointment results has the largest deviation compared to the other features. Many users encountered the same issue - activation of heat map does not navigate the calendar to the current date. This has caused many confusions and additional clicks to toggle back to the current date heatmap view if they are on a different week view. Based on these feedback, we have made some changes to improve users' experience in using the heat map by removing the unnecessary clicks (toggle to the current date). In the improved version, user will be brought to the current week's heatmap upon selecting "create new appointment" form.
Number of Errors
Based on the above chart, create and update appointment have the highest number of errors, 3 and 2 respectively. Many users encountered the same issue - Participants change dropdown selections before heat map finishes loading. Based on the feedback we have gotten, we have made improvement to prevent users from selecting the options in the dropdown too quickly.
Claim 1 Results
Taking the average of the component scores, the average mean score is 4.39 > 4. This supports our claim that participants find it easy to navigate the calendar and manage patients' appointments (CUD) effectively.
Some of the feedback/ comments made by the users are as follow:
- Interface is user-friendly, but one small thing is it’s a bit of a hassle to toggle between Dr. Ho and Dr Goh at some points of time
- Flow of entering information is natural and does not feel awkward. However, there must be a choice to save appointment without selecting mode of referral
- It is easy to identify the appointment types with the help of color coding
- I like how easy it is to switch between doctor's calendar and viewing the appointment types on the calendar
Claim 2 Results
Claim: Participants find the patient queue useful in helping them to track patients’ attendance (average rating: 4/5)
The number of clicks users took to perform the tasks are consistent with the average number of clicks by the members. This shows that the user interface is intuitive and easy to use. There is also no mis-interpretation of the various features in the queue page. Generally all the users found it easy to manage patient queue.
There are no errors in most of the tasks in the queue management module. The only error found was that the user could not associate the cross button to add to the no show list initially.
Overall the feedback gotten from this module was positive. The average score across all the tasks was 4.47 > 4, supporting our claim of participants finding the patient queue useful in helping them to track patients' attendance.
Below are some of the feedback from users:
- No show is good for keeping track of patient
- The patient queue page is very easy to use
- The no show feature helps save a lot of manual work required and time spent on compiling a list of no show list. The current work process requires us to manually check the appointments in the black book against the CMS and marketing form google spreadsheet that is managed by the marketing team. This is highly tedious and error prone.
Claim 3 Results
Claim: Participants find the marketing conversion and appointment scheduling dashboard beneficial in their performance analysis
For this claim, we are interested in finding out on the usability of the analytics module. Taking the average of the scores across different tasks, we have gotten an average of 3.975 < 4. This is slightly less than what we have claimed. Some of the reasons we have gathered that explained the scores given are as follow:
- Include feature for exporting to excel (table)
- Analytics dashboards are useful but some of the charts are quite difficult to interpret
Claim 4 Results
Claim: Participants like and have positive comments about our user interface (average rating: 4/5)
In general, users like our user interface and have positive feedback about it.
Collated Feedback Score - Survey Questionnaire
# | Question | Collated Comments |
---|---|---|
1 | What do you like about the application? | Very simple to use, simple to understand Will make us efficient Ease of use for essential tasks. Runs smoothly with no lag I like the analytics dashboards as it simplifies the current process where a lot of my work was done manually, across multiple channels (CMS, paper, online shared doc and black book). An integrated platform like this helps reduce my workload The app is easy to use. It will save us a lot of time especially in managing the appointments Very simple to use and many of the features built are what we need Very clear and nice Very easy to use. Reduces human error, making us more efficient |
2 | What do you not like about the application? | Disable auto swap? Maybe can prompt to swap appointment when you delete appointment on crowded days Learning curve for swap waiting list patients is steep Auto sending out of appointment reminder. Some patients might not like it Quite hard to understand and use the swap appointment function |
3 | Comment on how easy it is to use the application | Easy to navigate around and update appointments Application is intuitive very easy to navigate |
4 | Comment on the overall usefulness of the application in helping you complete your daily tasks? | Compared to the current work process, this application greatly reduces the time needed for us to complete some tasks such as calling patients to remind them of appointment, collating the patient no show list etc. Eliminates many of the manual work that we have been doing. Save us a lot of time and reduces error especially during peak periods. Maintains old function of our appointments book, and can be updated I like the analytics dashboards as it simplifies the current process where a lot of my work was done manually, across multiple channels (CMS, paper, online shared doc and black book). An integrated platform like this helps reduce my workload Streamlined work process Charts provided are useful as it allows drilling down of information Application reflects current functions but make a lot easier |
User Study Summary
- Overall, users find the application easy to user and intuitive to perform all required tasks
- Usability scores did not vary drastically from user to user
- Number of clicks are fairly consistent compared to the members
- Users are satisfied and have positive comments on all the features of the application
- User provided good feedback for improvement
User Study Takeaways
- Consistent usability score for different users, considering that different users have different functions to perform in their day-to-day operation.
- With users' feedback, we identified areas of improvement on these functions. Eg. export function.
- Allow us to evaluate the usefulness of these features. Team will improve on the features based on users feedback and manage all change requests (remove/ add feature). Most feedback has been documented and accounted for in our implementation plan in which the team will work on it in subsequent development.