HeaderSIS.jpg

IS480 Team wiki: 2015T1 4Sight Final Wiki

From IS480
Jump to navigation Jump to search
4Sight team logo.png
4Sight Home.png HOME   4Sight Team.png ABOUT US   4Sight Project overview.png PROJECT OVERVIEW   4Sight Project management.png PROJECT MANAGEMENT   4Sight Documentation.png DOCUMENTATION  
Main Wiki Midterm Wiki Final Wiki


Project Progress Summary

Click the following icons to access the respective document:

Presentation slides icon.pngFinals Slides Deployed link icon.pngDeployed Site Poster icon.pngPoster Pitch video icon.pngPitch Video

The credential to our application:
username: admin
password: 123

Development Progress

4Sight ProjectHighlights.png
Current Sprint: 13
Sprint Duration: 16 Nov 2015 - 27 Nov 2015
Upcoming Milestones:Final Presentation & Poster Day

Project Highlights

  • Completed 100% of our development work as of 21 Nov 2015
  • Application goes live on 28 September 2015 with actual users using our application to perform their day-to-day tasks. eg: scheduling of appointments, tracking conversion rate and marketing expenditures.
  • The project has officially been handed over to Clearvision on 26 Nov 2015.

Milestone Status


4Sight FinalMilestone Updates.png


Project Management

Project Status

4Sight Final Development Progress.png

Planned VS Actual Project Scope

Planned Actual

4Sight midterm scope.png

4Sight final scope.png

Major Scope Changes

What has been added?

Date Module Category Feature Description Value to Client
10/10/2015 Admin Module Secondary Periodic Appointment Backup Appointments are backup on a periodic basis - monthly (all appointments from the start of time to current day) and daily (one month worth of future dated appointments). An excel file containing the appointments detail will be sent out based on the configured timings. 1. Ensure continuity in business operations
2. Periodic backup helps ensure that no data is lost when the application crashes, serving as a safety net for the clinic.
24/10/2015 Analytics Module Primary Tracking of KPI The team will help keep track and provide real-time update on the monthly KPI. A successful increment on the monthly KPI occurs when a patient schedule for a surgery appointment and turns up for it. 1. Evaluate on the effectiveness of the marketing efforts
2. Monitor sales progress and allow for prompt actions to be taken
24/10/2015 Analytics Module Primary Forecast conversion based on the number of pre-evaluations scheduled Based on the number of existing pre-evaluations appointments, the team will forecast the number of surgeries that the clinic will be expecting for the current month/ upcoming months. This is done using linear regression based on historical data. 1. Allow for prompt actions to be taken. eg: if the number of surgeries forecasted for a particular month is low, the marketing team can invest more on marketing efforts to attract more patients, boosting sale.
2. Complement the tracking of KPI


What has been removed?

Date Module Category Feature Reason for Removal
10/10/2015 Mobile Calendar Module Tertiary Sync appointments to native calendar Good to have feature that is of no importance to the client. The chances of the doctors using this feature is very slim. Hence, the development cost in implementing this feature outweighs the value to sponsor.
24/10/2015 Admin Module Secondary clinic expansion sub-module 1. No crucial need for a scaling module and it is unlikely that they will ever use it.
2. Client prefers that we focus on improving the current features based on users' feedback.

For more details, please view our Change Management Here!

Planned VS Actual Project Schedule


4Sight midterm schedule.png

4Sight final schedule.png


Schedule Highlights

  • There are no major changes to project milestones.
  • Deployment was delayed by 2 days due to some technical difficulties faced. However, this did not affect the progress of the team.
  • Features are added (highlighted in yellow) or removed (strikethrough) to/from project schedule after careful deliberation by the team - Explanations can be found above, at the project scope section.
  • Slight modifications made to project timeline due to a delay in completion of tasks scheduled in iteration 11.

Schedule Metrics

View our Schedule Metrics Here!

4Sight FinalScheduleMetricScore.JPG

Schedule Metric Highlights

Sprint Planned Duration/days Actual Duration/days Schedule Metric Score Action Status
11 12 14 0.86 Estimates deviate from planned and team consumed 2 buffer days. Slight delay due to poor estimation of tasks hour as the team had underestimated the complexity of the customization of settings feature as part of the admin module. Project schedule was slightly affected and adjustments were made to ensure completion of project on time. Completed
12 12 14 0.86 Estimates deviate from planned and team consumed 2 buffer days. Some of the customization of setting feature tasks were carried forward to this sprint, causing a delay in the tasks that were originally scheduled for this sprint. Overall project schedule not affected. Completed

Tasks Metrics

View our Tasks Metrics Here!

4Sight TaskMetricScore.JPG

Tasks Metric Highlights

Sprint Planned Hours/hrs Actual Hours/hrs Task Metric Score Description Action Taken Status
11 59.0 72.0 0.82 Estimation of tasks is not as accurate as planned. There is an under-estimation of time required to complete tasks related to the customization setting feature of the admin module due to some changes requested from the sponsor. Team evaluated next sprint's tasks and made necessary adjustments to ensure that project schedule will not be affected. Completed

Bug Metrics

View our Bug Metrics Here!

4Sight BugMetricScore.JPG

Bug Distribution based on Priority

4Sight BugsByPriority.JPG

Bug distribution based on Severity

4Sight BugsBySeverity.JPG

Bug Metrics Highlights

Sprint Bug Score Summary of bug Action Taken
5 60 Most of the bugs were found during our deployment to heroku. The rest come from the interactions of the CRUD of appointment with other modules such as waiting list and queue. Team stopped all current development then and resolved the bugs immediately. All bugs were fixed within the scheduled time.
8 57 Most of the bugs found were from the analytics module. It derived from the complexity in the combination of charts we could have as well as the corner cases that we have not addressed in the analytics module. Team stopped all current development then and resolved the bugs immediately. All bugs were fixed within the scheduled time.

Project Risks

View our Risk Management Here!

Risk Type Risk Description Consequence Likelihood Impact Mitigation Strategy
Project Management Risk Unable to get actual users to conduct testing due to busy schedule Unable to get genuine feedback from actual users and hence affecting usability of system High High Coordinate in advance with Clearvision stakeholders so that they have ample time to gather relevant users for our User Testing, such as involving nurses and optometrists from other Clearvision clinics.
Technical Risk 2 way SMS might not work for all mobile device platforms (iPhone, Android etc.). Current provider, infobip might not work for iPhone. Team has to source for other providers that provide 2 way SMS service suitable for all mobile device platforms. This might cause a delay in project schedule Medium High Lead Developer to work closely with provider and check if 2 way SMS service is possible for iPhone. Meanwhile team has to look for alternative providers and ensure that it works well with all mobile device platforms.
Client Management Risk As the team is doing early deployment, there might be more change requests along the way since users will be using the application on a day-to-day basis. These changes might not be easily picked up in our demo done every sprints as well as the user testing we have conducted because it does not cover all possible scenarios. Incur higher development cost with increase change requests Medium High 1. Any proposed change request is to go through the change management process.

2. Inform Supervisor of any major change request.

Technical Complexities

Architecture Diagram

ArchitectureDiagram-deployment.png


Technical Complexity 1: Handling concurrent requests and slow clients

Issue

  • In prod, server of choice would be Gunicorn, adapted from unicorn from Ruby. Written in pure python, enabling quick deployment and ease of use with the Django framework.
  • However, Gunicorn as a Web Server Gateway Interface(WSGI) with a relatively small pool of workers (2x CPU Cores), can only handle a small number of concurrent requests.
  • If there is no buffering solution in front of Gunicorn serving static files, under large number of users, the Gunicorn WSGI will suffer from considerable lag, impacting usability.
  • Despite Heroku providing limited buffering functionality, large number of requests/responses will allow the Clearvision site to be susceptible to accidental/deliberate DDOS attacks.

Solution

  • A buffering reverse proxy needs to be used in front of Gunicorn to buffer requests and responses from the outside web to the Gunicorn WSGI.
  • Nginx is excellent at buffering slow clients
  • Nginx Architecture:
NginxArchitecture.png
  • Heroku does not support Nginx implementation out of the box. Therefore, we utilized a custom Nginx-Gunicorn buildpack for heroku deployment, instead of the regular python buildpack that Heroku uses for generic Python applications

Reasons

  • Gunicorn documentation strongly recommends Nginx in front of Gunicorn when in prod.
  • Fast serving of static content. Nginx forwards requests for dynamic content to Gunicorn WSGI when it is needed.

Outcomes

  • Application is now more resistant to DDOS.
  • When only static content is needed, application is faster than before.
  • Slow clients over the network are buffered, resulting in a more responsive application with multiple users.


Technical Complexity 2: Patient Volume Visualization

Issue

Often with the manual means of booking appointments, the admin assistant that actually executes the booking has to look though the appointment book, and count how many patients are on that particular appointment requested by the patient(on the call) before scheduling the patient. If the admin assistant feels that the appointment is approaching maximum capacity, she then decides to look for another appointment timing that has a lower patient count. The process then repeats; manually scanning through the dates(and weeks), just to look for a suitable timeslot to schedule the patient. 

This results in scheduled appointments that are not well spread out, leaving the clinic too busy - or too empty.

Solution

A consolidated view of appointment timeslot capacities deeply integrated with the main calendar to display scheduled applications. With this functionality, the user can, at a short glance, zoom in on appointment timeslots that are relatively empty - or full.

Upon initiating to schedule a new appointment, the admin assistant opens the "Add Appointment" drawer. Upon selecting the doctor and appointment type, the user is greeted with a plethora of timeslots from today onwards, colour coded to reflect how full(or empty) a particular timeslot is. 

Implementation Challenges

From a database perspective, appointments shown on the main calendar consists of appointments objects that exist with columns(date, time, doctor, patients, etc) - that is, appointments that have patients tied to it. 

In this case, how could we retrieve a list of appointments in the future that have no patients tied to the appointments? We figured that additional objects has to exist before the creation of appointments, and appointments had to reference these objects. This means that the additional objects not tied to any appointments are free(no patients).

Implementation

To tackle this issue, we broke up the appointment model’s columns(date, time, doctor, patients, etc) , to reference another table, namely Timeslots. Timeslot objects are configured upon load of the database, for a configurable duration (years), timing per appointment type and timing per doctor. In this way, appointment objects need not exist for us to recognise that there are no appointments tied to that particular Timeslot

For further customisation, the administrator has the ability to tweak the colours of the heatmap visualisation, according to the number of patients in the appointment timeslot.

In essence, a combination of timeslots that are tied to appointments and the timeslots that are not, form our heat map visualization.


Technical Complexity 3: Web Sockets

Issue

Despite having a relatively low rate of users using the application and executing the same action at the same time(think race conditions), we want to ensure that users are looking at the the most updated version of the calendar, enabling greater strategic decisions when scheduling an appointment.

When actions are done on one web instance, the same web instance updates the database with the new information. Technically, all clients have the latest information - but not until they force a refresh of the web page. To deliver a greater user experience, we want the web application to be self aware such that if one client makes changes to the database after a particular action, all other clients are aware of the change, and in turn update their respective pages, in real time

Solution

Websockets provide this functionality. Upon every action, the application sends out notifcations to all clients, and these notifications can be broken down into channels and events. In this way, our frontend client can listen on a specific channel and only update a certain part of the UI if its broadcasted on a particular event. For example, the frontend client can be listening on “AppointmentChanges” channel, but would only call the function to remove an appointment from the calendar when its received on the “RemoveAppointment” event.

Implementation Challenges

Although Django is a relatively old framework with the lack of websockets functionality built-in, its outstanding ORM and extensive documentation still made us choose to continue developing with it. Hence, we started our search for websocket libraries that provided tight integration with the Django ORM. 

We thought that SwampDragon was the answer. The library was working great in development. When database rows were changed, messages were pushed to all other clients through a dedicated SwampDragon server, over a redis broker.

Then we tried to deploy it on our deployment server, Heroku, and that when a lot of time was spent trying to make our real time application work. 

To cut the long story short, the Choot Jail Heroku architecture disallowed running a server process alongside the main web process. Even if SwampDragon was running on another web application, it would further complicate matters because we had to decide which dyno should be running the redis broker process, and it may introduce cross domain problems. We went further to attempt a workaround to run two processes(web and SwampDragon server) on one dyno, and using Nginx as a reverse proxy to route requests from a subdomain of our application to the SwampDragon server, to no avail.

Reasons for implementation

After countless attempts and time spent trying to solve the issue, we finally chose the Pusher API as the framework to deliver this functionality. With a allotment of 100,000 messages per 24 hour period on the free tier, it is more than sufficient for our application’s scale.

Methods in the backend are configured to trigger notifications to the Pusher server, and in turn the Pusher server pushes the notifications in the channel-event standard to all other clients. AngularJS in the front end then listens on the specific channel-event standards, and updates the calendar/queue accordingly.


Technical Complexity 4: Logic behind the calculation of conversion

From a technical perspective, the process of modelling how many patients have converted with respect to each marketing channel is relatively complex. This data is then used to calculate what is the Return on Investment on each marketing channel.

As seen in our application, there is an option to add in additional marketing channels, as and when the marketing personnel decides to. However, the approach to calculate ROI for marketing channels is a tricky situation, as the effectiveness of a particular marketing channel might only surface in later months, and not in the month that the marketing dollars was spent.

To illustrate the logic, a simple flow of the lifecycle of appointment types that a large proportion of patients go through is shown below:

LogicDiagram.png

Each conversion is calculated when a patient comes for any appointment before surgery and then eventually attends surgery. Hence, if a patient comes for a Screening/Pre Evaluation before a surgery and eventually comes for a surgery, the conversion count is tagged to the former appointment.

Therefore, the ROI per marketing is calculated by the number of patients that “converted” to surgery appointments multiplied by the surgery cost, and divided by the total marketing dollars spent.

As a result of this business logic, we have integrated additional layers of logic to ensure that when users leverage the custom ROI filter, the channels that are available are filtered based on the initial selection of the start and end dates. This is to ensure that the channels that the users see are not the entire range of channels, but those that were spent in the date range specified.

Quality of Product

Intermediate Deliverable

Stage Specification Module
Project Management Minutes Meeting Minutes
Project Timeline Project Timeline
Project Scope Project Scope
Metrics Schedule Metric
Task Metric
Bug Metric
Risks & Mitigations Risk Management
Sponsor Change Requests Change Management
Requirements User Stories Link to all user stories
Analysis Diagrams Use-case Diagram
ER Diagram
As-is Process
To-be Process
Market Research Market Research
Design Mid-Fidelity Prototype Wireframe
Testing User Test Plan and Results & Analysis User Testing 1
User Testing 2
User Testing 3

Deployment

View our architecture diagram for deployment here!

Testing

Our team has conducted 3 user testing with the actual users of our application.

User Testing 1

Location: Clearvision @ 6 Nutmeg Road
Date & Time: 23 August 2015, 10:00am
No. of Participants: 3
Objectives:
1. Verify that functionalities built are in line with user requirements
2. Determine if the user interface is intuitive
3. Gather feedback on current functionalities
4. Identify usability problems

View our User Testing 1 Here!

User Testing 2

Location: Clearvision @ 6 Nutmeg Road
Date & Time: 16 September 2015, 15:00pm
No. of Participants: 8
Objectives:
1. Verify that functionalities built are in line with user requirements
2. Determine if the user interface is intuitive
3. Gather feedback on current functionalities
4. Identify usability problems

View our User Testing 2 Here!

User Testing 3

Location: Clearvision @ 6 Nutmeg Road
Date & Time: 16 November 2015, 15:00pm
No. of Participants: 3
Objectives:
1. Determine usability of the admin and analytics module
2. Verify if features built are in line with user requirements (admin & analytics modules)

View our User Testing 3 Here!

Reflection

Team's Reflection

4Sight reflection.png

Member's Reflection


Team4Sight Final LearningOutcomes.png

Sponsor's Comment

4Sight sponsorComment.png