HeaderSIS.jpg

IS480 Team wiki: 2015T1 4Sight User Testing 3

From IS480
Jump to navigation Jump to search
4Sight team logo.png
4Sight Home.png HOME   4Sight Team.png ABOUT US   4Sight Project overview.png PROJECT OVERVIEW   4Sight Project management.png PROJECT MANAGEMENT   4Sight Documentation.png DOCUMENTATION  
Technical Diagrams Design Documents Meeting Minutes User Testings Project Documents

Venue

  • Location: Clearvision @ 6 Nutmeg Road
  • Date: 16 November 2015 (Wednesday)
  • Time: 15:00pm

Objectives

1. Determine usability of the admin and analytics module 2. Verify if features built are in line with user requirements (admin & analytics modules)

Claims

Our team made 4 claims -
1. Participants find it easy to navigate the calendar and manage patients’ appointments (CUD) effectively (average rating: 4/5)
2. Participants find the patient queue useful in helping them track patient’s attendance (average rating: 3.5/5)
3. Participants find the marketing conversion and appointment scheduling dashboard beneficial in their performance analysis (average rating: 4/5)
4. Participants like and have positive comments about our user interface (average rating: 4/5)

Participants

  • Number of participants: 8
  • Roles of participants: Optometrist, admin clerk and marketing team member


IMG 1365.JPG IMG 1358.JPG IMG 1353.JPG IMG 1367.JPG

Tasks

Each of the users was given a list of tasks to complete. Click here to download tasklist. The tasklist also consists of some questionnaire to gather feedback from users.

Summary of Survey Results

Click here to download the raw data.

Usability Results

4Sight UT2 UsabilityScore.png



4Sight UT2 UsabilityScore2.png


Usability Analysis

  • No drastic variation in the usability scores between users of different roles
  • Simple UI that allows user to perform fundamental tasks easily
  • Confirmed the value of proposed features such as swap list
  • Discover potential improvement eg: Block off doctor’s time slots on calendar

Claim 1 Results

Claim: Participants find it easy to navigate the calendar and manage patients' appointments (CUD) effectively (average rating: 4/5)

Number of Clicks


4Sight UT2 Claim1NoOfClicks.png


Based on the above chart, create appointment results has the largest deviation compared to the other features. Many users encountered the same issue - activation of heat map does not navigate the calendar to the current date. This has caused many confusions and additional clicks to toggle back to the current date heatmap view if they are on a different week view. Based on these feedback, we have made some changes to improve users' experience in using the heat map by removing the unnecessary clicks (toggle to the current date). In the improved version, user will be brought to the current week's heatmap upon selecting "create new appointment" form.


4Sight UT2 claim1Changes.png


Number of Errors

4Sight UT2 Claim1NoOfErrors.png


Based on the above chart, create and update appointment have the highest number of errors, 3 and 2 respectively. Many users encountered the same issue - Participants change dropdown selections before heat map finishes loading. Based on the feedback we have gotten, we have made improvement to prevent users from selecting the options in the dropdown too quickly.

4Sight UT2 claim2Changes.png


Claim 1 Results


4Sight UT2 claim1Score.png


Taking the average of the component scores, the average mean score is 4.39 > 4. This supports our claim that participants find it easy to navigate the calendar and manage patients' appointments (CUD) effectively.


Some of the feedback/ comments made by the users are as follow:

  • Interface is user-friendly, but one small thing is it’s a bit of a hassle to toggle between Dr. Ho and Dr Goh at some points of time
  • Flow of entering information is natural and does not feel awkward. However, there must be a choice to save appointment without selecting mode of referral
  • It is easy to identify the appointment types with the help of color coding
  • I like how easy it is to switch between doctor's calendar and viewing the appointment types on the calendar


Claim 2 Results

Claim: Participants find the patient queue useful in helping them to track patients’ attendance (average rating: 4/5)

4Sight UT2 Claim2NoOfClicks.png


The number of clicks users took to perform the tasks are consistent with the average number of clicks by the members. This shows that the user interface is intuitive and easy to use. There is also no mis-interpretation of the various features in the queue page. Generally all the users found it easy to manage patient queue.

4Sight UT2 Claim2NoOfErrors.png


There are no errors in most of the tasks in the queue management module. The only error found was that the user could not associate the cross button to add to the no show list initially.

4Sight UT2 claim2Score.png


Overall the feedback gotten from this module was positive. The average score across all the tasks was 4.47 > 4, supporting our claim of participants finding the patient queue useful in helping them to track patients' attendance.

Below are some of the feedback from users:

  • No show is good for keeping track of patient
  • The patient queue page is very easy to use
  • The no show feature helps save a lot of manual work required and time spent on compiling a list of no show list. The current work process requires us to manually check the appointments in the black book against the CMS and marketing form google spreadsheet that is managed by the marketing team. This is highly tedious and error prone.

Claim 3 Results

Claim: Participants find the marketing conversion and appointment scheduling dashboard beneficial in their performance analysis

4Sight UT2 claim3Score.png


For this claim, we are interested in finding out on the usability of the analytics module. Taking the average of the scores across different tasks, we have gotten an average of 3.975 < 4. This is slightly less than what we have claimed. Some of the reasons we have gathered that explained the scores given are as follow:

  • Include feature for exporting to excel (table)
  • Analytics dashboards are useful but some of the charts are quite difficult to interpret

Claim 4 Results

Claim: Participants like and have positive comments about our user interface (average rating: 4/5)

4Sight UT2 claim4Score.png


In general, users like our user interface and have positive feedback about it.

Collated Feedback Score - Survey Questionnaire

# Question Collated Comments
1 What do you like about the application? Very simple to use, simple to understand
Will make us efficient
Ease of use for essential tasks. Runs smoothly with no lag
I like the analytics dashboards as it simplifies the current process where a lot of my work was done manually, across multiple channels (CMS, paper, online shared doc and black book). An integrated platform like this helps reduce my workload
The app is easy to use. It will save us a lot of time especially in managing the appointments
Very simple to use and many of the features built are what we need
Very clear and nice
Very easy to use. Reduces human error, making us more efficient
2 What do you not like about the application? Disable auto swap? Maybe can prompt to swap appointment when you delete appointment on crowded days
Learning curve for swap waiting list patients is steep
Auto sending out of appointment reminder. Some patients might not like it
Quite hard to understand and use the swap appointment function
3 Comment on how easy it is to use the application Easy to navigate around and update appointments
Application is intuitive
very easy to navigate
4 Comment on the overall usefulness of the application in helping you complete your daily tasks? Compared to the current work process, this application greatly reduces the time needed for us to complete some tasks such as calling patients to remind them of appointment, collating the patient no show list etc.
Eliminates many of the manual work that we have been doing. Save us a lot of time and reduces error especially during peak periods.
Maintains old function of our appointments book, and can be updated
I like the analytics dashboards as it simplifies the current process where a lot of my work was done manually, across multiple channels (CMS, paper, online shared doc and black book). An integrated platform like this helps reduce my workload
Streamlined work process
Charts provided are useful as it allows drilling down of information
Application reflects current functions but make a lot easier

User Study Summary

  • Overall, users find the application easy to user and intuitive to perform all required tasks
  • Usability scores did not vary drastically from user to user
  • Number of clicks are fairly consistent compared to the members
  • Users are satisfied and have positive comments on all the features of the application
  • User provided good feedback for improvement

User Study Takeaways

  • Consistent usability score for different users, considering that different users have different functions to perform in their day-to-day operation.
  • With users' feedback, we identified areas of improvement on these functions. Eg. export function.
  • Allow us to evaluate the usefulness of these features. Team will improve on the features based on users feedback and manage all change requests (remove/ add feature). Most feedback has been documented and accounted for in our implementation plan in which the team will work on it in subsequent development.