IS480 Team wiki: 2015T2 spirITus Finals Wiki
- View our final presentation slides here!
- Visit our deployed application [connectus.das.org.sg here!]
- List of changes after midterms:
- User can remain anonymous in forum changed to functionality of audience visibility
- User experience enhancements
- Deployment to a DAS server and conducting live trial
- Project Challenges:
- Learning curve for mass email notifications was higher than expected due to the usage of a message queue to give users a faster browsing experience.
- The change of POC (Point of contact) from Jeanne to Sree resulted in many changes. While the functionalities descriptions are mostly the same, it is the nitty gritty details such as how make up lesson are to be handled and stuffs that caused lots of changes to our database schema and codebase.
- Mass emailing of parents when publishing announcements turns out to be more complex than expected as we attempted at improving the user browsing experience through the use of a message queue to reduce wait times for the teachers.
- Project Achievement:
- Secured pay services for free during this fyp project.
- The application demo during midterm presentation was disrupted and was the 1st of it’s kind due to the suspension by Twilio & a mistake in the configuration of IronMQ (We forgot to remove the previous subscribe urls resulting in too many subscribers to the queue). This combination resulted in the server crashing. However, we managed to bring back the server fast enough and continue with the demonstration proper.
- Though our IS480 was not smooth sailing initially, We managed to stick through thick and thin as a team despite through effective communication and mutual respect and understanding.
- Being able to see how our sponsor appreciate our effort in the detailed development in the UI and the utility of the application. It is almost similar to see the final outcome of the structure we built together, starting with a lego block, brick by brick towards it's completion.
- Managed to get our application deployed on 1 of DAS in house server (which is currently on a laptop in DAS office)
|Iteration||Module||Functions / Comments||Confidence|
|1||Admin and accounts UI||UI using jQuery mobile. Coincides with IDP||1|
|2||Account Module||Teacher account can View, create and search||1|
|Parent Account UI for reports, events and chats||1|
|3||Accounts Module||Creation of profiles. Entity profiling||1|
|Classroom Module||Teachers can mark attendance, guardians can view child's attendance||1|
|Feedback Module 1||Teachers can mark attendance, guardians can view child's attendance||1|
|4||Feedback Module 1||Guardians can view child's education progress||1|
|Communication Module 1||Teachers and parents are able to send message to each other||1|
|Search and Filter Module||Teachers can search for student by name||1|
|5||Accounts Module 1||Admin account have CRUD access, able to reset password||1|
|Classroom Module||Teachers, students and guardians can view lessons/ scheduled plans||1|
|Communication Module 2||Teachers can create announcements for class. Guardian can get notifications on child's progress||1|
|Rewards Module||Teachers can award points to students and student can view their awarded points||1|
|6||Desktop rollout||Desktop UI and Accounts CRUD implementation||1|
|Rewards Module||Gamification techniques and concepts changes to make it more engaging.||1|
|Feedback Module 1||Teachers can generate student profile, attendance statistics including assessments||1|
|7||Forum Module||All users can post queries in forum||1|
|Feedback Module 2||Student can view their attendance statistics||1|
|Teachers' dashboard showing class activities and student reports(Desktop)||1|
|8||Feedback Module 2||Teachers can opt to send generated reports to email accounts||1|
|9||UI Enhancement, Validations||UI Enhancement and more comprehensive validations for Mobile and Desktop site||1|
Our team has made very minimal changes to schedule, most of the changes were implemented during the planned iterations. The most notable changes were the change in functionality requirement, the communication module. Our client conducted their own trials and wanted a 'contact us' form for parents to communicate with the teachers. We were informed to focus more on the mobile development in December 2014 as DAS wanted to use our application in January for trials.
Dated: December 2014
Dated: April 2015
|Iteration||Planned Days||Actual Days||Score||Remarks / Actions Taken|
|1||35||35||100%||None. We are on schedule.|
|2||21||26||81%||Reasons: Learning curve in learning new frameworks and integrating libraries resulted in slower-than expected-progress in development.|
Extend iteration till the task is completed. Adjust the dates of the subsequent iteration to match the use of buffer days.
|3||21||27||78%||Reasons: All members are still learning the new language and framework. Coupled with other academic commitments, the difficulty in developing back-end logic for student attendance-taking in PHP resulted in minor delays in task completion.|
Extend iteration till the task is completed. Adjust the dates of the subsequent iteration to match the use of buffer days.
|4||21||20||105%||None. We are on schedule.|
|5||14||15||93%||Reasons: Our designer left the group, and existing work had to be reassigned to other members.|
Used one buffer day to complete planned functionality in iteration. No need for adjustment of scope.
|6||14||16||88%||Reasons: Change of POC, handover of management to Sree. There were also new suggestions of additional functionalities by the directors, which led to some major changes in the database structure.|
Used two buffer days to complete planned functionality + new requirements in iteration. Extend iteration till the tasks are completed. Adjust the dates of the subsequent iteration to match the use of buffer days.
|7||14||14||100%||None. We are on schedule.|
|8||14||12||117%||Reasons: Finished 2 days earlier of planned days as this iteration coincides with midterm preparation which results in many late nights and development progress.|
Bring forward the next iteration and start accordingly.
|9||12||12||100%||None. We are on schedule.|
|10||12||12||100%||None. We are on schedule.|
|11||12||12||100%||None. We are on schedule.|
|Iteration||Low||High||Critical||Total Score||Actions Taken|
|1||0||0||0||0||No actions required.|
|2||1||0||0||1||Used debugging time to debug.|
|3||4||1||2||29||Used debugging time + one buffer day to debug.|
|4||1||3||1||26||Bugs were fixed during testing and debugging time. No buffer time required.|
|5||4||1||0||9||Bugs were fixed during testing and debugging time.|
|6||8||4||0||28||Used debugging time to debug. One outstanding UI bug to be fixed in later iteration due to low importance. Used one buffer day to debug.|
|7||4||1||1||19||Bugs were fixed during testing and debugging time.|
|8||5||4||0||25||Some UI bugs were found during Sponsor Test 2; Most bugs resolved during testing and debugging time, in preparation of Mid-terms demonstration.|
|9||2||1||2||27||Bugs were fixed during testing and debugging time.|
|10||5||2||0||15||Bugs were fixed during testing and debugging time.|
|11||5||1||0||10||Bugs were fixed during testing and debugging time.|
|1||PDF Generation of reports||The challenge lies in the transformation of database records data to the required pdf display format. The team has also spent lots of time in getting certain special symbols to be displayed on the pdfs as well with font embedding.etc. It was an issue which took a long time (as nobody seem to know the correct solution) before someone online knew of a solution to it since we first asked it online.|
|2||Attendance Reports||The challenge lies in the transformation of database records data to the required pdf display format as well. Also lots of "hacks" are used to get the things displayed exactly as wanted on the PDF due to quirks in the PDF rendering (just like how different web browsers could display something differently). The handling of how to indicate the make up lesson dependencies (lesson A is make up for lesson B is for C.etc) is also complicated.|
|3||Lesson Feedback and makeup||There are many client changes as to how lesson feedback and make ups are to be recorded. Resulting in frequent database schema and code changes.|
|4||Speeding up of notifications (mass emails, sms.etc)||To reduce the wait of teachers when publishing new announcements to guardians. The team has to learn about how to use a message queue. This is quite a challenge as more than 1 member have yet taken the Enterprise Integration module yet.|
Software Quality Attributes
Performance / Scalability
File Minification and Compression
File Minification involves the removal of extraneous whitespace (Line breaks, indentation spaces, tabs) and code comments as shown in the pictures below. Thus resulting in a smaller view file for users to download. While this may not sound like much of a saving at 1st glance, the speed and bandwidth savings does adds up when you have more site visitors over time over many pages. Minification is generally a good practice used by many sites and was recommended as one of the quick wins one can make when it comes to optimizing site performance. By removing extraneous whitespace, it also helps to highlight/resolve issues whereby the layout of UI elements look out of place due to unintentional white spaces that went undetected by human eyes.
Furthermore as most Singapore telcos currently only offer 2GB of data currently. Being reasonably considerate to our users is something we thrive to achieve.
Use of Message Queue for Time Consuming Tasks
Certain tasks such as the mass sending of emails when publishing post can take up quite a lot of time. To reduce the wait on the users on such tasks. Our team has made use of IronMQ Message Queue such that the user will not need to wait for everything to complete as the job is first queued. The free tier of IronMQ allows up to 1 million API requests per month which is more than enough messages for DAS needs. To deal with the message size limit of 64kb of the free plan, our team only send the post_id to IronMQ then when IronMQ hit our queue receiver url, we will then grab the post content from the database. Hence allowing for post content which may be very long.
Eager Loading only when it helps
Eager loading is a technique whereby related entities are loaded as part of a query. A benefit of this technique is such that the amount of queries could be greatly reduced. Thus resulting in a speed boost as Laravel ORM's Eloquent, which uses lazy loading by default could end up taking too many trips to the database to retrieve what it needs. The downside of it is that sometimes it may give you more data than needed. Thus our team only applies eager loading when the benefit clearly outweighs the downsides.
In terms of security, by using a modern web framework such as Laravel, many of the common security vulnerabilities/features (e.g Bcrypt password hashing.etc) are being taken care of by the framework as long as we do things properly. Form Inputs are also properly validated and escaped as necessary to our best known efforts. Given that a good portion of our users involves young students, Our team has decided to implement rate limiting as well on areas related to the emailing of DAS staffs.
Using the Laravel web framework has helped to impose certain good practices and structure on our codebase. Such as the use of composer to manage backend package dependencies, migrations for database structure changes, Eloquent as an object relational mapper to access the database.etc.
However for something specific to our project and not coming right out of Laravel is how we handle the issue of having a separate mobile and desktop site as we did not go for a responsive web design approach due to having partially done our website with jQuery mobile as part of IDP (We also do not want to be constrained by some of the possible limitations of responsive web design on our site's layout as we to be able to better satisfy our client's requirements and UI changes requests). Our team thought about how we could best return a mobile or desktop view response cleanly and our solution to that problem involve the detection of the visitor's UserAgent in the BaseController's constructor. Then in the subcontrollers when we return the appropriate view by appending the string 'desktop' or 'mobile' cleanly as shown below. Avoiding the need to use many if statements.
After resolving the above issue, there was however still this maintainability issue that there is often a need to display an object's field differently in the views. Such as the formatting of an announcement published datetime. Repeating this kind of logic in many view files is not good for maintainability. This problem is exacerbated further for our situation as we are having separate mobile (jQuery Mobile) and desktop (bootstrap) sites. To deal with this issue our team decided to go with the concept of View Presenters. This way display logic code for how to format certain object's fields are being kept in 1 place. Making it more maintainable as well as adhering to the Single Responsibility Principles as it is not really the job of the model classes to store display related logic as the Laravel community recommends. Our team has decided to use laracasts/Presenter to help us in the implementation of it (View Presenters)
In terms of accessibility, other than having proper error pages than to throw an unfriendly browser default 404, 500s error pages. The team also make sure for critical services, there are proper backup plans such as more than 1 email service provider (by setting up Mandrill and Mailgun accounts in advance) or queue backends. This way in the event if something is gone or no longer supported (Say if Mandrill were to drop it's free plan). DAS can still switch to the 2nd option and continue operations.
|Steep learning curve in exploring new frameworks, libraries and programming languages, such as SMS integration, E-mail sending, the Laravel framework and generation of PDF reports||High||High||We will seek help from web resources (tutorials, videos) on how to implement it in our application. After many hours of effort and studying, we gradually became more familiar with the use of such libraries and frameworks.|
|Sudden changes in requirements, such as the communication methods available using the application, which arose after analysis of the results of their own trials||High||High||We will seek advice from our supervisor and inform her about the new changes. We will proceed to implement the changes if it does not delay the completion of the project.|
|Juggling our academic workload in tandem with FYP when there are difficult tasks in a particular iteration||Medium||Low||We will seek advice from seniors and read up on best practices on how we can utilize the framework chosen to make coding more efficient. Working long hours into the night was common as we tried not to exceed the planned iteration days.|
|There is a possibility that we may send out too many emails and get banned temporarily by Gmail, if we exceed their daily email sending limits (2000 for premium accounts, 100-500 for normal accounts)||Medium||Medium||Make use of a reputable transactional e-mail service such as Mandrill, which allows up to 12,000 free emails per month. The team has also set up Mailgun, an alternative e-mail service such that in the unlikely event that we exceed Mandrill's free limits for a month, the team will also be able to switch to Mailgun without much hassle. It offers 10,000 free emails per month, which isn't too bad as well.|
|Project Management||Meeting Minutes||Meeting Minutes|
|Schedule and Bug Metrics||Metrics|
|Project Scope||Scope / Functionalities|
|Analysis||Architecture Diagrams||Architecture Diagrams|
|ER Diagram||ER Diagram|
|Use Case||Use Case Diagram|
|Testing||User Acceptance Test & Sponsor Test Results||Test Results|
- Exploring a new language and framework
- how the use of technologies solve helps in solving business problems
- Handling a project involving a business entity and managing it well
- Balancing expectations.
- Effective time Management juggling IS480 commitments with other projects etc.
- Conflict Management and resolution strategies
- Staying positive and always have the end in mind
Project Manager: Tan Zheng Ming, Kenneth
- Effective communication between stakeholders involved in project
- Manage conflict and expectations effectively
- Planning of technical tasks is often complex and subjected to changes
- Establish effective collaboration by allocating tasks / resources appropriately
- Plan ahead and execute accordingly without procrastination
Lead Designer: Chua Chong Thee
- Designing is a subjective experience that relies on a lot of intuition and inspiration to formulate a design that can please add many people as possible. After spending so much time on Bootstrap and its intricacies, I now appreciate good web design better, for I understand that thought and effort required to make it happen.
- Perfectionism is an important trait to have, for it makes us question ourselves and pushes us on to do better.
- Google is a really powerful tool when used properly, and I have lost count of the times it has saved me during my design expedition.
- It is important to be steadfast in our decisions, but not inflexible. We must be able to rationalize our own decisions when questioned, but overfixation on something is just denying ourselves of the endless possibilities that exist. Always be open to suggestions, for there may be better ideas out there just waiting to be discovered.
Lead Developer: Lan Ziming
- Learning when to loosen control and when to tighten it.
- If something feel complex, chances are there is a much simpler solution
- See beyond the surface, Sometimes I wanted to use a service but it's either paid or comes with a very limiting free plan as stated on their website. However when I decide to email some of them telling them that I'm a student or doing it for a social cause, I'm surprised that some are actually willing to let me use their paid plans for free just because I'm a student or doing something for a social cause.
- I have also learn to be more people orientated. As IT students, we often think about doing what's technically the most impressive. However it is also important to consider people and context. The best solution is often not what best fit the problem theoretically . But one that also consider the people and conditions around you (team mates, clients, supervisor, context.etc).
- Lastly, I learnt that resting and thinking of better ways of doing things is very important. So as to go farther and faster. Working longer hours is not the solution to everything.
System Analyst: Chen Weifang
- Adapting and working with members with different working styles
- Breaking down the algorithm into smallest parts possible and be meticulous
- The obvious solution is not always the simplest solution.
- Picking up a new language is not complex as long as one has the interest to learn
- Understanding user’s reasoning is important in determine the priority of functions
QA Analyst: Chan Qianru
[My FYP Journey is coming to an end! Yippie!]
After almost 1 whole year of planning, preparation, developing and testing of our application, I’m glad to say that our team has finally accomplished all our milestones for FYP. Our FYP journey, in short, was a tiring, tedious, demanding and stressful phase of SIS life where work seemed never-ending. However, we have overcome all our obstacles and emerged as a strong team and even stronger individuals. Here’s what I’ve gained and learned from FYP:
1. I learned how to code many new languages The scope of our FYP project requires our team to learn new frameworks (Laravel) and languages (PHP, SQL, HTML etc). Those who knew me in school know that coding and I don’t match (unfortunately ). Despite my initial weak coding foundations, my team has been very patient and helpful in guiding me on better coding practices. For example, in the earlier iterations of FYP, I was assigned easier functions to build up my coding experience and familiarity with Laravel and PHP. I also did pair-programming with stronger coders, who will help to oversee my codes and make sure that I don’t accidentally introduce bugs into our system. All these took time, effort and perseverance on my team’s part. Ultimately, I also learned to reference and do my research from coding forums and discussions. As our application grew, my coding skills and coding confidence grew, too.
2. We take turns to support each other during busy periods There were peak periods where we had floods of presentations, midterms, tests and report due dates. Thankfully, our team has learned how to give and take by having the initiative to do a larger share of work when we know our teammates is quite tied down with other commitments. Afterwards, when we ourselves are busy, the rest will help to take over some of the FYP tasks. (Reallocation of tasks is not uncommon.)
3. I learn how to meet expectations, and exceed them too Throughout various sponsor meetings, user testings and sponsor feedback, I learned how to convey their needs and expectations into our application. It could be in the form of better code functionalities, nicer UI designs or clearer instructions. By getting regular and frequent feedback from users and testers, I am more confident that our application can meet the expectations of our sponsor. Also, by learning how to code better as I get more familiar with the codes, I was better able to meet the deadlines of coding tasks. I’m sure the overall quality of my codes improved too! (Thus exceeding the team’s expectations of my codes). Some of my key takeaways from the user testings include: Knowing what the end users want is important to solve any problem. Good designs often make people have a lasting impression of our application, while responsiveness and real-time will make systems more user friendly.
Some funny FYP incidents upon reflection:
- Bugs are afraid of Ziming: whenever I found a bug in the system, I would consult my group mates on how to resolve it. However, whenever I replicated the process of creating the error to Ziming, the page(s) worked perfectly ok! It was quite puzzling at first (how the bugs magically disappeared whenever I showed my laptop screen to Ziming but reappeared for other members), but ultimately our group concluded that the bugs are afraid of Ziming (a.k.a the bug-killer) and are in hiding.
- Sleepy codes: I once (or twice) fell asleep while coding online with Kelly via Teamviewer and Skype. We were trying to resolve some code functions of lesson feedback, and I fell asleep unintentionally. Kelly tried to wake me up over the Skype call but failed. To be fair, it was already 2+am in the morning.
- Changing development environment is beneficial to our team: By development environment, we mean the environment where our team come together to code. Over the past few months, our team would sometimes meet in other polytechnics for some coding and creativity inspiration. They are mostly quite conducive for our application development.