|Home||The Team||Project Overview||Project Management||Project Documentation||Project Resources|
- 1 Project Progress Summary
- 2 Project Management
- 3 User Test
- 4 Technical Complexities
- 5 Learning Outcome
- 6 X-FACTOR
Project Progress Summary
- Following are the summary of the new scope added during the course of project.
- Login -Login feature is to allow user to have control over their setting and data. Previously without this feature, users of the system ended up overriding previous settings.
- User management(admin) - This includes the basic CRUD features(Create,read,update and delete). This feature is to allow our client to give access to users whom are interested in the product.
- Treatment Configuration(admin) - This is allow the admin to key in the tests that will be populated in user's dropdown as well as to select tests that he/she wants to appear as default in user's profile.This is to ensure users do not re-enter tests name and enters in legitimate tests.
- Datatable view of text file - Instead of uploading text file for data feeding,this new in-built excel type data table in our system allows users to copy paste data in.This feature would no longer require our client to share sensitive data file to users.Everything is in-built now !
- Run all strategies-comparison report -Previously,users could only run one strategy at a time with their parameters.That means they have to run 4 different simulations if they wanted to pick the best out of the 4 strategies. We wanted to enable users to run all the strategies at a single button and show them the results of it.Users can then pick the best strategy at a glance.
- Favourites - During a dialogue session with our client,we realised our client typically uses a particular setting over a period of time to run simulations.In order to save her from entering in the parameters such us test types and data reapeatedly ,the favourites feature was created to help her save time.One click and all the previous data be populated accordingly.
- Seeds configuration checkbox - Seeds play a big part in simulation as it affects the reproducibility and fairness of simulation results. With the seeds currently being hardcoded, this new feature gives users to fix/unfix random seeds.
- Significant time spent in database changeover and in exploring visualisation libraries
- Database was changed from MySQL to MongoDB - the reason for this changeover was we realised the bulk of the data we are storing are non-relational in nature, and therefore, it does make more sense in using a document-oriented database over a relational database. In addition, as the amount of data that will be stored are large, there is also the significance performance saving in terms of each write and read operations.
- Due to the steep learning of D3 data visualization library, team explores other alternatives such as Highchart, Chart.js, nvd3, Google Chart before finally settling on C3.js for data visualization.
- User interface went 4 major overhauls
- After each user test and internal review,we get feedback and suggestions to improve the flow.Right from the start,our aim was to give the best UI possible to our client.From portal based UI to step by step approach UI during midterms,we have finally designed a aesthetically pleasing UI that relates with users intuitively.Users can jump tabs and run simulation at a click.
- Simulation modelling
- The purpose of this animation is to allow non-technical users to visualise and see the bottleneck at a glance.User can playback to any timeframe and find out about patients status(happy,sad,satisfied),type of patients(entrant or retrants),their position and queue size at the hospital.
- Reporting tools
- Each of the 4 charts presented here are the key performance indicators healthcare practitioners would look out for.We want our charts to help them in their decision making and based on feedback from our users we have happy to have met that goal.
- Real users
- We were lucky to get healthcare practitioners as testers for our User test.One of the main takeaway was that the system is far from maturity.Hospitals would want a realistic setting instead of current randomness.However,they are willing and have shown interest to take on this project for further development.
Click on this link to view our full project schedule .
Planned Vs. Actual project plan.
The chart above show the overview of our project schedule, which we were slightly delayed in 3 different iterations marked by the red regions; namely iteration 5, iteration 7 and iteration 10. For every delay in our iterations, we will need to push our schedule slightly behind by a day or two, highlighted in the regions colored in purple. How we manage our schedule would be to use our breaks (colored in yellow) interchangeably with our buffer days. This means that the more buffer days we use the less number of days for breaks we will have. In total we have 13 buffer days, from the start of the school term to the last day of our project (poster day).
During iteration 5, we saw a 1 day delay due to the change of our database from MySQL to MongoDB. There was a little difficulty with our integration with MongoDB as we are more accustomed to the syntax used in MySQL, which we use it more often when doing other projects.
We had to stretch the iteration by 2 more days. One of the reason was because the industry users (in the hospital) had limited available dates to test our application. Another reason was due to a bug in our application, which appeared when we tried to implement other queue strategies in addition to the FIFO strategy we had for our simulation. To revive the crashed system, we took longer in the implementation / integration of the queue strategies than expected.
Our schedule was delayed by 1 day, mainly because our user test had to stretch across a few dates so that we can gather different users (professors, student, hospital practitioners/ staff) to help us test the application. In addition, all the members were very busy in meeting deadlines for other modules due in week 13 and week 14 of the school term.
Project Iteration Overview - Summarised List of Completed Tasks
Refer to our schedule planned Vs. actual for the overview. The delays results in scores under 100, but were not major that would require functions to be dropped. Metric score for iteration 11 is a forecast value, as we are confident our project will end smoothly on poster day.
For the detailed log of our bugs recorded on Mantis Bug Tracker, you can access this link with the credentials provided:
Username: guest Password: guest$1
Responsible. Assist. Consulted. Informed.
RACI Matrix - we use this metric so that every member in the team is in-charge-of at least one major module or task throughout the entire project. This helped to organise the way we keep one another updated with our progress and be more efficient during meetings. How it helps us to be more efficient is when, we can work simultaneously in smaller groups on tasks categorised by the modules in the table.
Refer to the first responsibility in the table:
For example, Basith as the Project Manager is responsible for organising meetings with supervisor, sponsor and industry users for application testing. Clarice will assist him in checking the internal deadlines for the project. And he will have to consult Chun Yang, our lead developer, if our development progress is good before we present our work to the stakeholders. Syafi, who is the business analyst needs to make make sure our application on production side is running well (90% bugs free). Not forgetting to always involve everyone in the team, Nigel and Faris will be kept in the loop, while they can still focus on their coding tasks.
Risk Assessment & Mitigation
The following are the summary of our risk we faced during the course of project:
1: : Unable to acquire data for predictive modelling
As expected due to strict PDPA controls over sensitive data,our client was not able to provide us with the huge chunks of data for analysis purposes.Because we already forecast this possibility,it enabled us to quickly drop the predictive model and move onto prescriptive models which we had brainstormed to cover our scope in the event of no data.
2:: First hand experience on building simulator
The team faced problems debugging errors the simulator was producing.Uncle google was not able to provide us with much help ,hence the team relatively took more than planned time to.
3:: No prior knowledge on MongoDB
We allocated a team member to research on MondoDB before implementation period.While implementating this team member was able to share his expertise and mentored fellow team members.Despite this effort,this new unexplored database took us more than the planned time causing the iteration to delay.
4::New semantic UI(new framework)
Initially the team was skeptical with Semantic due to its low maturity,but the library was powerful enough to support us with the functions we needed.
5::Tasks were not assigned based on competency and availability
Even though the team members were taking 4 modules in average,some members had other commitments such as Inter-University sports competition.This ate into their time and as a result hindered them from completing thier tasks on time.Also,team members were not able to cope with the complexity involved in their assigned task.To tackle this,a informal dialogue chat is held at the start of every meeting to update PM on their personnal schedule and preference.This helps the PM to allocate task more appropriately.
6::Increase in the number of inputs leading to scope creep
We have been getting alot of suggestions from healthcare practitioners and the team wants to use this suggestions to improve the projects.The team is overwhelmed with ideas and has the tendency to attempt implementation of such ideas since its coming from the industry.However,the team discussed with the client and implemented those within capacity.
7::Tendency to implement subsets of complex functions
The team came out with ideas to implement subsets of a complex model due to time constraints.However,such a model might not meet user needs and the feature might not be worth the time.The team took time to discuss and the PM made sure such a simplified model meets users needs before development
8::Underestimate effort required to playback function
The client needs a playback function that will enable to fast forward and backwards her simulation.Again this domain is unexplored and team might struggle with the complexity.To tackle this,we immediately started on this function after midterms.So even if it delays,we still would have time to complete it.
9::Increasing workload towards the end of the semester
Despite the huge commitment and time FYP takes,we students cannot afford to lose focus on other modules as a result.The peak weeks are from week 10 onwards and students generally get busy with other projects.In view of this,the PM identified each members deadlines and tailored tasks individually.For instance ,The PM pushed faris to complete all his tasks by week 11 since he had 3 deadlines on week 12.
Goals of User testing 3:
1) Professors and Healthcare Industry Personnel: To evaluate the current state of the system (simulation model, reporting tool, and playback features). Gather feedback on how to further develop the application to make it an application that is mature enough for practical adoption by local hospitals as part of a project in the future.
2) Students: To test vigorously on the usability of the application based on “Nielsen’s 10 Heuristics” so as to improve the user experience of the application.
Some of our participants:
User test 3 was conducted mainly between 3 groups of users.This were the main results from the test.
|Group of Users||Profile||Type of testing||Main feedback|
|Students||Consists of 10 SIS students who have or is currently taking IDP||Students were given study tasks to do and they are asked to evaluate mainly on the usability of the system based on “Nielsen’s 10 Heuristics”||1.Student’s find it hard to understand the tasks. Context and explanation has to be given. However, base on our observations, the student testers is able to complete the tasks rather easily. |
2.Gave several constructive feedback on the usability and also, recommended changes to improve the UI. Team has incorporated it to the list of changes to be implemented.
|Healthcare Industry Personnel||Consists of healthcare practitioners from SGH and another local hospital||Testers were asked to evaluate the current state of the application and how the application needs to be further developed to be mature enough for practical adoption||1.Excellent feeback on the interface and usability of the application. They like the clean layout of the various parts of the application (simulation parameters, results page etc)|
2.However, more can be done to validate the model. Suggest doing multi-site/hospital studies to validate the model. Willing to collaborate with future teams on this project to provide relevant data from SGH on this aspect
3.Members of the local hospotal suggested showing results on “Door to doctor” and having “95th percentile and 50th Percentile” as these are additional things that the hospitals normally track. Overall, He is able to visualize how this application will be able to help their “Specialist Outpatient Clinic” operations
|Non-Domain Experts||Professor Lee Wee Leong, Professor Venky and Professor Lian Chee.||Generic feedback on system's usability and model of which the simulation is built upon.||1.Minor usability comments to improve the UI. Team has already incorporated it to the list of changes to be implemented. |
2.Questions the model of simulation, especially on distribution of patients and the treatments that they are ordered to undergo. Suggest using past data to provide a more accurate model here.
3.Recommends an export function for the simulation results to allow further analysis on MS excel.
List of changes to be implemented after User Test 3
|Minor UI changes to be implemented:||1.Changing the “Arrival rates” from the tabs to “Patients Arrival Rates”. In addition, the “Arrval Rate” header in the data table to be “Arrival rate(average)” |
2.‘Next button’ to navigate around the simulation parameter tabs.
3.Take away the “Day” column in the data table as it is redundant, since there are day tabs below.
4.Minimizing the size of the graphs
5.Axes labels from numbers to day names
|Other changes:||1.Export the result from simulation to a csv format file for further analysis by users of application|
2.Displaying a ‘Door-to-Doctor’ graph in results page.
3.Change the Length-Of-Stay(LOS) chart to a line chart and the Resource Utilization chart to a bar chart in the results page.
4.Change the option of toogling percentiles to 95th and 50th percentile.
Takeaway from the User Test 3 :
1) The current state of the user interface and layout of the results page is in good condition base on the comments from student testers, professors and industry personnel.
2) Due to the lack of historical and existing patient data from hospitals to implement a more accurate framework of assigning the treatments to the various patients and also, to validate the whole model, it is important to acknowledge that this application will continue to be a work-in-progress towards practical adoption.
3) Going forward, SGH has expressed interest in collaborating with future groups on improving the model by providing them with the necessary data. The application and the model can then be further developed upon and refined to be mature enough to be used by the local hospitals.
Understanding Simulation Model
This is undoubtable one of the most complex challenges the team has faced when working on the project. All members are novice in the world of simulation and have absolutely no experiences. Therefore, much effort was required to be spent on understanding the following questions:
- a.How does simulation work?
- b.What is the difference between Discrete Event Simulation and other types of simulation?
- c.How do we model one programmatically?
- d.How does one calculate the randomness factor using various kind of statistical probability distribution using historical data?
Usage of Design Patterns
Design pattern was used in this project to achieve maintainability, flexibility and scalability of our code architecture. On the server-side, DAO design pattern such as the Abstract Factory pattern was used to achieve easy migration of data source in the future, if there is a need to.
Hacking 3rd party framework and libraries
Calculation of coordinates for animation
Due to the way SVG animation works, co-ordinates for the final position are not stored for the object being animated, rather they stored the transform_x and transform_y of the object, which is how many pixel have the object being moved X and moved Y. Therefore, calculations have to be made to find out where the object is on the screen at the point before it is being animated. Positions for the objects are also dynamic, which means object should try not to overlap with each other.
Chun Yang - This project gave me the platform to push myself.I had the opportunities to try out new technologies to build onto the development capabilities that I was already possessed with. The nature of this project was different from what we were taught and that gave me a tough yet worthwhile experience
Faris - This is480 journey is definitely a memorable one. Personally, I've learned a lot of interesting things in both technical and non-technical aspects. For the technical side, one of the takeaways is about data visualization principles where we learnt how to present data in the most effective way. As for the non-technical side, I've gained alot of experience on client and supervisor management - on how we present our progress as well as gathering of feedback and requirements. Also, through this project, I was given the opportunity to explore new skills such as designing a poster and also developing a video pitch. We've finally come to the end of the project and though it is not near maturity, what's important is that we know we've put in our best to make it a success.
Nigel - Uncertainties are the only constant in this project. Be it in the development or project management phase, we all have to deal with it. I believe that i am coping much better with uncertainties compared to day one of this project. i think we have managed to mitigate some of these uncertainties by being extremely critical and reviewing our project frequently from the perspectives of our clients and other stakeholders.
Clarice - As a group, we have learnt to adapt well meeting requirements from the sponsor and initial changes to improve the application, which helped to create the application we have today.
Syafi - I’m amazed at how a project has grown in ways which we never really anticipated as a team. So many angles have been considered in terms of the project’s business capabilities!
Basith - Apart from keeping your supervisor and client happy,one has to make sure the team is happy ,driven and motivated as well. Practice flexibility,give them the space and breaks and most importantly read between the lines.After all,like machines humans can break down too!!So Project management is not just about having great plans, proper metrics and meeting deadlines.
Our group has successfully developed an simulation and reporting tool tailored to the needs of local hospitals.
When Less is More
Currently, most established simulation software (examples provided above) are able to provide users with ample features even in their most basic package. However, most of the features are not utilised by the hospitals as they do not require them. To quote the head of healthcare analytic department at a local hospital, he praised us for building an application which is easier to use with features specifically cater to the needs of the hospitals. With only the essential features, users will not be overwhelmed by the links, buttons and ribbons, but they would be able to navigate and run their simulations easily.
Potential of our Application
Though our application still has rooms for improvement, we managed to lower the learning curve of using a simulation tool. This would be especially useful for doctors and healthcare practitioners whom are not trained or apt at using such tools. Hence, there is a lot of potential for our application to be adopted by the local hospitals.
Lastly, our application is unique as it is equipped with different queuing strategies (FIFO, SCON, SREM, MIXED) that can be used to simulate patients' activities at the ambulatory care, where their queue priorities are dynamically managed based on the algorithms. And special credits goes to or sponsor (Prof. Tan Kar Way), for the algorithms are based on her research paper.