Team Brownie Points Final Wiki
Project Progress Summary
- 1 Project Progress Summary ★
- 2 Project Management ★
- 3 Quality of Product ★
- 4 Reflections ★
- 5 Slides ★
Project Progress Summary ★
To date we have completed all our 5 planned phases and 3 iterations for the project on schedule, although there were initially some delays in schedule due to timetable conflicts and midterm examinations. We also managed to deploy our system live by 20th November 2011, before our final presentation, so both the client and our team were able to view it live. We have also conducted 3 UAT and continuous testing after each change, throughout the course of the project and made improvements to the system based on feedback received. Our client is pleased with our system.
- Since midterm presentation, we brainstormed and increased our connectivity functions to include a Control Panel; its function was to serve as a Connectivity Monitor – At a glance, the Control Panel presents the Users that are not connected, the Users’ Giving Growth and Businesses’ & Users’ Continuity. Zooming in, B1G1 personnel can simply click on the users displayed to bring up all of the users' details such as contact number, past giving projects, giving impact, B1G1's last contact with the user, email open rate. Upon viewing this, B1G1 personnel can then easily connect with users via Communicare’s web SMS service or any other method and log this easily in the user's details so that the data is updated centrally for the next person to view.
- We also expanded the Trends and statistics function – Its purpose is to allow B1G1 to filter and trend the connectivity of different groups of users according to Country, Industry, Membership type etc, over specified periods of time. This allows B1G1 to compare and analyze giving trend for different users. We also came up with an idea that allows B1G1 personnel to compare the trends and charts across categories. For instance, to compare the giving trends and statistics of Category A users: from Accounting industry in Australia, whose last giving date was less than 1 year ago; to Category B users: from Finance industry in Australia, whose last giving date was less than 1 year ago; to Category C users: from IT industry in Australia whose last giving date was less than 1 year ago; all the client has to do is replicate the filter panel many times while changing the Industry (or other) variable and many charts that show the trends across the Australian industries will be shown in the same page!
- We also expanded our AutoEmailer function - While previously its purpose was to send automated emails to users, we have ensured that the email and report content is now very customized and personalized and that the emails are automatically sent to the right users at the right time designed to encourage giving behavior, for instance, we made sure the emails would be sent according to different timezones in different countries. We then implemented monitoring for tracking and follow up purposes - once sent out, these messages are tracked and their statistics such as click through rate are analyzed to facilitate decision making process. We also used Bitly as discussed later, to track the location and timing of the emails opened.
Close communication with client
- We worked closely with the client. meeting in person at least once every one to two weeks, and communicating via email and Skype nearly every day. Doing so allowed us to discuss changes and seek the client's opinion and obtain continual input and feedback. We also made sure that we made sure to deploy the system by 20th November 2011, which is 3 weeks before our final presentation, so that we would be able to see ready to use by 20th November 2011, one week before our final presentation. In addition, we came up with a more efficient way of communicating with the client - instead of the system sending weekly messages, we created a report that would be sent to the B1G1 administrator that would highlight the changes that need to be taken. As the system would be used in a real life business scenario for B1G1, it made it all the more important to us to make it as good as possible.
Comparison of main features from Acceptance and Midterm stage
- We worked closely with our client and our supervisor who have been very supportive and constantly give constructive feedbacks to see improvements in our project scope as well as features and usability. The change from one phase to another was drastic and it consumed a significant amount of time debugging and improving the functionalities as well as usability aspects.
Below are the notable features that had been improved throughout the project duration from acceptance stage to final stage.
- We were constantly challenged by the change in scope of our project and addition of new features which were never thought of including during the acceptance stage. This meant we had to finish more work within a shorter period of time than expected. Refining the new features also took a lot of time as we had to brainstorm how to make them more functional and user-friendly for the client. We mitigated this by having more discussion during our meetings and brainstorming and preparing fully before each meeting so that we could generate better ideas together.
- We also faced difficulty in predicting the schedule and time allocation for the newly added functions as these were not planned for at acceptance. This made it necessary to adjust and update our schedule frequently and this got more difficult as the term progressed and our workload got heavier. We mitigated this issue by asking expert opinion on the members who had prior experience working on such functions in other projects (e.g., Visual Analytics).
- Successful tracking of email responses: We were able to track and measure the email responses from users so that we could measure their connectivity to B1G1 and thus highlight to B1G1 if it needed to take action to encourage greater giving behavior.
- Familiarized ourselves with complex drupal database structure: Our project uses Drupal database and its structure was inherently complex and took a lot of time to understand, especially since B1G1's database was very messy as well. After much effort and time, we were able to familiarize ourselves with the structure to build our system.
- More accurate forecast in time allocation for known tasks: We gained experience in scheduling and planning the duration of tasks based on complexity. Initially we were too optimistic and planned too short durations but we soon learnt to be more realistic and give careful thought to details so that we would not have to keep pushing back tasks if we fell behind.
- Managed to work closely with a client in a real business scenario: We communicated very closely with our client and met every week in person and skyped or emailed nearly every day to obtain constant feedback so that we could make our system better. We kept her aware of what we were doing and made sure she was very involved in each development.
- We did a lot of research and communicated with the client to benchmark our system against commercialized systems. We then recommended new features to the client although the client did not think them necessary at present. Benchmarking
Project Management ★
Project Schedule (Planned vs. Actual)
- Now, we have come to the end our of project. We have completed the project and prepared for the deliverables required. And we are in the preparation stage for the IS480 poster day.
- One notable change from midterm schedule is that we have the third round of UAT, which served the purpose of not being able to tested the required functionalities to the right people. During UAT 1, due to data confidentiality, we could only test the penultimate phase with 3 B1G1 members, including Masami. For UAT2, we tested the non-confidential features of our project to 20 SMU students, including 2 technicians. Therefore, UAT 3 served the purpose of testing the right functionalities to right people.
(Details on UAT tests and results can be seen below).
Click here for the detailed schedule breakdown and metrics.
- Happiness Metric
- The Happiness Metric measures the team's and individual's happiness level through out the project. Low levels of happiness contribute to low working optimality. We take actions such as having a short coffee break, group sharing, having lunch break to increase happiness levels and thus boost productivity. Here, the scores are trended across the following 2 graphs.
- The team’s average happiness level was higher during Jul, Aug and September.
- After Mid-term presentation, the happiness level fell significantly to 2.33.
- Nearing the end of our project, it increased slightly to 2.52. Exam stress also contributed to the low happiness level.
- The individual’s average happiness level by month showed Su to have the highest happiness level.
- Wann and Lynn had the happiness level with least variation. This was significant because Wann and Lynn are the PM and assistant PM and they need to motivate the team by remaining optimistic
- The actual Happiness Metric file can be viewed at the following link: Happiness Metric
- Drupal Database
Although the database is hosted on a standard mysql server, the database itself is structured in a Drupal schema, which requires a lot of data manipulation effort to retrieve desired data. The queries were more complex but what we are concerned with the most is the querying time and its effect on overall system efficiency. We explored some of the possible solutions we can do to increase this efficiency such as SQL Server Integration Services and MySQL stored procedures but they were not compatible with our system or our database.
Therefore, we decided to use simplified tables that we populate and update daily with a daily data processor to simplify the query and reduce the querying time for the system.
- Integration of Fusion Charts
The person responsible for the integrating of fusion chart has to research from various resources, compare and analyze different implementation methods and figure out the best way to integrate into the current development framework. And there were issues with local server and the deployment server.
- Configuration of Deployment Server
Because of the use of Pear Packages for html bulk email sending, we have to configure the server to install this package, which requires some research because deploying on a shared web host means there is no direct access to php server configuration file: php.ini
- Tracking of Individual Email from Auto-Emailer System
Quality of Product ★
|Project Management||Minutes||Meeting Records and Minutes|
|Metrics||Schedule Metric, Happiness Metric|
|Analysis||Use case||Use Case Diagram|
|System Architecture Diagram||System Architecture Diagram|
|Screen Shots||Communicare System|
|Testing||Test Plan, UAT2 Final Results,UAT3 Final Results,Final Testing Page|
|Handover||User Manual||User Manual|
|Code||Client FTP Server, Deployed Project|
Stability: Our maximum email output from our system is 500 emails. We had tried sending this from the live deployed server to test its stability and there are no known issues.
Performance: Although there are slower server-side oriented and query-heavy modules, we have configured so that they will be efficient enough for clients' needs. Please see Technical Complexity No.1 and 3 for more information of this configuration.
Usability: Various usability tests with participants including students from SMU, IT professionals and B1G1 board members are carried out towards the end of the project to ensure the usability of the system for all types of users.
Maintainability: We will be documenting a developer's guide and be ready to help if necessary for B1G1 to maintain the system.
Our finished system is deployed here at: Communicare note: We are not able to give out the log in information due to confidentiality issue.
- From start to finish, we conducted 3 UAT and after UAT 3, continuous testing each time a change was made.
Please see Test Plan for the test plan.
UAT 2 (20.09.2011 - 27.09.2011)
- In UAT2, we at first encountered difficulty in getting the client to consent to having other people test the system due to confidentiality reasons but we expressed that we would not be getting sufficient feedback to make it a better system with only testing done by the client. So after much persuasion, we got the client to consent to having 20 SMU students and IT Professionals test the basic functions that would not allow them to view customer data and trends and statistics.
- We made UAT2 to be task-based because we assumed that most people will not read the user manual first before using the system so we wanted to make sure it was very user-friendly. We used Quantitative (Average time taken in min, Number of Clicks taken, Difficulty Ranking (1-least, 5-most) & Qualitative measures (Areas of Difficulty, How we can make this better) in UAT 2 to gain as much feedback as possible and track the duration and number of clicks it would take to complete each task.
- Usability Testing: In UAT 2, we personally observed Ms. Masami test the usability of our system. We provided tasks for her to complete without reading the user manual and monitored how long and how easily she was able to complete the tasks. We also incorporated the changes based on her feedback and the difficulty she encountered in completing the tasks. Of the 13 tasks that we set for her, she was able to complete 5 with difficulty ( Difficulty ranking of 3 and more out of 5 - highest) and 8 reasonably easily (Difficulty ranking less than 3).
- Email SIT Testing: In UAT2, we tested if the emails were sent to the correct user, if the content is correct and how it looks in different servers (gmail, mac mail, yahoo, outlook)
- Testing: We did testing on all devices to make sure that it's compatible with all devices. So, we did testing on Mac, Android, iPhone, Windows and Outlook which is the most problematic so we included a new link that will take the user to a new browser to get around the Outlook problem.
- Functionality Testing: We talked to the client and planned the testing so that we can test on the real server and test it in existing capacity. We also discussed the risks of testing on the real server with the client. Therefore we made the decision to test during time zones in the UK and Australia over the weekend when there was less user traffic from existing customers so that even if the system crashed the impact would be minimized. It was difficult to coordinate this as the time zones in the UK are Australia are 7 hours apart and the UK is 7 hours behind Singapore.
- After editing, we plan to take it a step further in UAT3, by focusing on the timing by testing if the correct email is sent to the correct user at the correct TIME.
For the actual results file for UAT 2, please see the following link: UAT2 Final Results
- We conducted UAT3 with 10 members of the B1G1 board at their board room at Raffles Place.
- Again, we used Quantitative (Average Time taken, Average number of clicks, Difficulty ranking (1 - least, 5 - most) and Qualitative (Areas of Difficulty, How we can make this better) Measures.
- On the whole, UAT3 results were much better than UAT2 results. The difficulty ranking dropped significantly, the average time and number of clicks taken was also reduced.
For the actual results file for UAT 3, please see the following link: UAT3 Final Results
UAT 2 VS UAT 3
- Due to the quantitative measures used in both UAT 2 and UAT 3, we were able to benchmark the Email Trigger and Poll functions to measure how much impact our adjustments from UAT 2 had. The following graphs show the difference in Average time taken in minutes, Average Number of clicks and Difficulty Ranking to complete the task. As you can see, the results in these 3 categories have improved significantly.
As a team, we learnt the following:
- Project management skills – How to plan and adjust our schedules to meet changing requirements, mitigate risks, use team metrics to improve the quality of our work and 3 UAT to constantly make the system more user friendly.
- Communication skills - How to manage client expectations, communicate well within the team, demonstrate our learning process, challenges and how we overcame them to supervisors.
- Teamwork - How to overcome conflict and leverage each other’s strengths to create synergy to value add to our project.
- Technical skills- How to architect our databases, user interface and emailer system to go above and beyond meeting the current needs of the client by allowing for future expansion and scalability.
Aung Thu Wann:
- As a project manager, it is very important to have a clear communication while working with the team members, clients and supervisor. Our team had gone through several changes in the project scope and it is the best if we can communicate efficiently. By efficient communication, it means that we must have a clear and open communication to achieve success. We cannot make assumptions and always good to double confirm. Our usage of social media, for instance, Facebook secret group feature, helped a lot.
- There can be unforseen changes and clashes in the schedule even if we plan carefully from the beginning. For instance, the school's new policy on Deepavali holiday made cancellation of classes and shifts in the presentations, subsequently followed up a series of schedule clashes. Thus, it is important for the team to always have small buffer times in every week to cover up the delay in schedule in the even that such clashes occur.
- It is very important that one handles the workload effectively and not to bring non-FYP related work or mind to the FYP table of meetings or group work. There were times when I experienced conglomerated workload from projects, studies and CCA commitments. However, I had to constantly keep in mind that having many commitments is never an excuse to work less on FYP.
- From the experience of 4 months of course work and a whole summer of preparation, I learned a lot in areas of both technical aspects of the project and personal development by dealing with the business client in a real world situation.
- If I can go back to the beginning (inception stage) of our project, I would effectively communicate with my team members and spend time to learn from them.
Chew Lee Chen:
- As a developer, the greatest take away is learning that while it is important that the functionality works, but perhaps, it is MORE important for the system to be user friendly and aesthetically pleasing to the eyes. The clients probably do not mind having one less function, if the rest looks professional and easy to work with. As in real life, look matters. For a system with poor user interface and poor aesthetic, people will judge it to be inferior and of lower performance than what it actually can do.
- Working with client makes me realized that communication between different parties is vital. After all, client is not developer and could not foresee what they really want, until they see the system and feedback on it. As such, I realized that even though we accomplished what the client had wanted initially, client would have a change of mind and decide to make amendment. Without proper communication, the system will always fall short of expectation.
Lynn Tan Min-li:
- As lead tester, I learnt that the client usually has confidentiality issues as the client will not want the company data exposed to other people but that we should not let this stop us from looking for ways to improve the system. For instance, in our UAT 2, after much negotiation with the client, we were able to test functions without exposing company data, with 20 SMU students and IT professionals.
- As a lead tester, I also learnt that when the test form is detailed and requires thought from the people testing the system, it is important to sit down with them and request each time that they fill in all the answers to your questions and put thought behind it. Otherwise, they are prone to leaving blanks or not giving much thought to answers when there are many questions.
- I also realized the importance of communicating a lot within the team and with our supervisor and client. This helped us to generate better ideas and obtain more feedback.
- As assistant PM, I learnt the importance of documentation so you can track progress and use that to set the agenda for each meeting so that everyone can come prepared and plan ahead so the project can be finished on time.
Ma Myat Noe Mon:
- As a developer, I had learned how to deliver not only the expected but far and improved systems to the clients within the expected time limits. Client communication is also quite important especially when the project is like ours and the clients end up being the users.
- Usability of the actual user might be different from the developer's perspective. During various stages of UATs, I had realized that what I had thought the optimal way doesn't necessarily translate into what is the best for the users and you should always get more opinions and suggestions on how to improve your systems.
- Technically, I had learned in terms of both programming skills and system architecture design skills. One thing I noticed is the need to constantly review your original system architecture throughout the whole project duration and revise as necessary. This kind of architecture management is what we couldn't have learned in a normal school project with stable scope.
- Team management is also quite challenging because trying to sorting out team problems and resolving conflicts among the team can be quite delicate to handle and you should always be prepared to talk the problem out rather than hiding it and hoping it will go away.
- All in all, I would say that, without having to attend a single class, I had learned a lot of technical, project-related and people skills from FYP.
Su Myat Mon:
- I have learnt that it is important for the team to be clear with the project scope and clients' requirements. To do so, the team, especially those who will be implementing the functionalities, should work very closely with the client since beginning. Only then, the team will have better picture of what kind of system client wants and can save time on implementing extra functionalities that are beyond client's requirements and satisfaction.
- Moreover, it is important for the developers to leave a buffer time before the actual delivery date (have to get all the functionalities done before the ). Since no one can forsee what kind of risks are likely to occur before the deadline, the team, especially the developers, should be ready to overcome any risk faced if they have enough buffer time before deadline.
- After all, for me, FYP experience is my first-time working with real client and has taught me a lot about the difference between school projects and the real-life projects, where more complex business logics, larger impacts of the system, etc. are involved.
"Communicare is a system developed by the team Brownie Points to enhance our organisation's activities. It allows us to communicate with our users in much more personal ways. It also gives us new understanding of the complex data generated by our existing systems. And finally, it gives us the ability to take effective actions - simply and immediately.
It's been a challenging project for the team since we have a complex data structure and a very unique operation model. But the team has been very open and willing to take on more challenges. And because of that, we now have something that is way beyond out initial expectation.
We really look forward to using this system each and everyday to monitor our progress and create more giving." - Masami Sato, Founder, Buy1Give1
Click here for the powerpoint slides.