Difference between revisions of "IS480 Team wiki: 2012T1 Bumblebee Final Wiki"

From IS480
Jump to navigation Jump to search
Line 300: Line 300:
===<div style="color: #ffffff; font-size:0%">Methodology</div>===
{| class="wikitable" cellpadding="15"
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Methodology</b>
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
====User Testing Environment====
Computer platform : Intel Pentium Processor
Screen resolution : 1028 x 768
Operating System : Windows XP
Set-up required : Computer date format (English (Australia)) of d/MM/YYYY
The participants will attempt to complete a set of scenarios presented to them and to provide feedback regarding the usability and acceptability of the application.
1. Kevin Choy, SATS Airline Relations Manager - Person-in-charge for this project.
2. Goh Wei Xuan, SATS Airline Relations Manager
These instructions were given to our clients:
1. Each user will be accompanied by 1 facilitator.
2. Users are encouraged to verbalize their movements, purpose, and problems.
3. Facilitators will record mistakes and questions made by users during testing.
4. To start the test, click on the file named “START.bat” found in folder named “SATS_Bumblebee_Beta_v5”.
5. All sample files needed for testing are found in:  SATS_Bumblebee_Beta_v5/data
6. Database used to store imported data is also found in ROOT folder.
7. Users are allowed to change their input(s) to verify data validity.
8. Users are to complete the tasks stated below. After completing each task, users have to answer the test questions pertaining to the specific task.
These are the task descriptions given to clients:
Below are tasks for users to complete.
1. Bootstrap/import files(s)
This task is for user to import data from excel files such as Flight Schedule Departure, Flight Schedule Arrival, Staff Records, etc. into the application. The application will then use these data for simulation purpose in the later step.
2. Add staff costs
This task is to record various costs in hiring staff into the application.
3. Add uncertainties
This task is to record the mean and standard deviation of different uncertainties that will affect the initial schedule prepared by the application. Simulation period = 7 days (represents the number of days the data is to be generate).
4. Run simulation
Run simulation to start assigning staff to different job assignments.
5. View staff schedule (in Gantt Chart)
This allows user to view and compare between a staff’s planned and actual working time.
6. Add airline requirements
Airlines have several different requirements on number of CSA and CSO needed.
This task is to record the individual requirements into the database. The input data will be used for simulation purpose in the later step.
7. Generate result
This task is to view the result generated in PDF format.
====Team Roles====
Overall in-charge (Yosin Anggusti)
- Provide training overview prior to usability testing
- Defines usability and purpose of usability testing to participants
Facilitators (Glorya Marie, Suriyanti)
- Evaluate on the application and user interaction with the application, rather than evaluating on the user
- Facilitator will observe and enter user behavior and user comments.
- Responds to participant’s requests for assistance
Test Observers (Yosin Anggusti)
- Silent observers
- Assists the data logger in identifying problems, concerns, coding bugs, and procedural errors
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Midterm_Wiki#Top|Back to Top]]</u></font></div>
===<div style="color: #ffffff; font-size:0%">Usability Metrics</div>===
===<div style="color: #ffffff; font-size:0%">Usability Metrics</div>===

Revision as of 18:21, 26 November 2012

Back to Main Page Project Overview Project Scope User Testing 2 Heuristic Evaluation

Project Management

Learning Outcome and Reflection

Final Presentation Slides




Click here to download our final presentation slides.

Project Overview



The Bees



B Sponsor.jpg

Project Description

Project Description

Singapore Airport Terminal Services (SATS) is the leading provider of the gateway services and food solutions in the region. It has a staff of 800 to handle more than 35 airlines. Every year, there are two major airline flight schedule change - summer (April) and winter (November). Each change can be a drastic one and has great impact on staff roster. One of SATS’s goals for its staff roster is to meet all its airlines’ requirements. However, there are many uncertainties such as staff members falling sick, staff resigning and flight delay, that cause the roster planned to be ineffective. Thus, Duty Manager in SATS often has to make last minute changes to the rosters and incur costs such as Staff Recall Cost, Over-Time cost and Meal Allowance Compensation (MAC). To help improve the efficiency involving various costs and consequences of the planned staff roster, our project aims to create a Staff Deployment Simulation Software (SDSS) which will first deploy staff based on flight schedule, flight requirements, staff records and staff roster and then simulate the roster plan by taking into account all the various uncertainties and forecast the various costs that the management would have to incur. Understanding the cost and consequences of a given roster, SATS would then be able to make necessary adjustments to avoid high expenses to the company. The diagram below explains the flow of the software.


To predict the uncertainties, we will take in the mean and standard deviation of the listed simulation parameters in diagram above(MC Rate, Flight Delay Rate, Staff Resign Rate, etc). Having those values, our software will then generate a normal distribution of each uncertainty. After such, we will be applying the uncertainties at the planned roster plan such as removing staff from the roster plan if he/she is taking MC. The software will also try to mimic the actions of Duty Manager who does the necessary adjustment to the staff roster to fulfill the airlines requirements i.e. to imitate human decision making process to come out with an optimal staff roster.

At the end of the day, our software will generate a management report which will reveal the cost and consequences of implementation of a given roster. The output given will be Total Staff Working Hours, Flight Demand Coverage, Meal Allowance Compensation (MAC), Over Time (OT) Cost, OT hours, Staff Utilization Rate, Unproductive Hours, Recall Cost and Recall Hours. Understanding the cost and consequences of a given roster, SATS would then be able to make necessary adjustment to avoid high expenses and thus, increase cost savings to the company.

X Factor

X Factor


Our system is able to handle a highly complex, huge domain size, scheduling problem efficiently in comparison to SATS’ current manual scheduling process.

Algorithmic Complexity

Optimal Scheduling

We provide our client an optimized staff schedule to effectively reduce client's overheads from non-optimal scheduling practices. In essence, we developed an optimal scheduling algorithm to maximise SATS’ resources to meet all airline requirements. This optimal scheduling is written based on Greedy Algorithm concepts.

Probabilistic Simulation

Our system gives our client a clear perspective of cost loss profile from human resource planning uncertainties (flight delays, staff resign, etc). And we would provide recommendation for minimizing additional cost effectively. To be specific, we use normal distribution and Kolmogorov-Smirnov test to forecast the uncertainties which SATS could face. After which we run a simulation to plot our client’s cost profile.

Aviation Industry Knowledge

Our system complies with numerous aviation specific business rules. To develop this system, our group has to understand aviation industry practises deeply. For this, We did a 2 months weekly attachment to SATS' Duty Manager(DM) and Customer Service Officer(CSO).

To understand the complexity and the abundance in probabilities of scheduling, try to solve this:
We want to color countries on a map so that no two adjacent areas have the same color. How many colors do you need at the most? Map Coloring Teaser
Courtesy of IS103 Computational Thinking



There are 2 main motivations for working on this project:
1. The industry that we are working with - it's aviation industry, in which we have minimum exposure in school! We would love to explore and understand more about what aviation industry is and how IT can help to make a difference in it!
2. Our software can make a significant difference as it will be implemented in SATS' daily operation. =D

Project Scope

Use Case

Use Case
Use Case Diagram v4.jpg

Deliverables and Scope

Deliverables and Scope
S/N Component Modules Description Requirements
1 Bootstrap Data Import data: flight schedule, staff records and roster plan from CSV files to the application database. Support multiple format (txt,csv).

Notify user if there is an error in importing.

2 Manage Airline Requirements Set special requirement such as how many CSA, CSO needed for a combined gate for each airline. Able to cater to different combination of flight sizes.
3 Pre-deploy staff Schedule staff roster based on employees’ working hours, system qualifications and other factors. Application development methodology should be agile iterative process coupled with test-driven development.
4 Perform simulation for deployment plan Perform simulation for optimized flight schedule, staff records, flight requirements and roster plan by mimicking Duty Manager’s activities such as calling back staff and extending over-time hours. Include all generated uncertainties/parameters in the simulation.
5 Measure the effectiveness of the roster Measure the effectiveness of the roster by simulating the Unproductive Time, Airline Demand Coverage, Over-time Cost, Staff Recall Cost, Meal Allowance Cost, and many others. Simulation results should be credible and realistic.
6 Manage past results Save and view past results. Past results could be exported as PDF report file. GUI should be user-friendly and simple

Please refer to use case description for greater details.

Graphical User Interface

Graphical User Interface

User Testing 2

User Testing 2, 25 October 2012

User Testing 2, 25 October 2012

The goals and objectives of usability testing

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed

Click here to download our report

Usability Metrics

Usability Metrics

Critical Errors

Critical errors are deviations of results from the actual result. These errors will cause the task to fail. Facilitators are to records there critical errors.

Non-Critical Errors

Non-critical errors are usually procedural, in which the participant does not complete a task in the most optimal means (e.g. excessive steps, initially selecting the wrong function, attempting to edit an un-editable field). These errors may not be detected by the user himself. Facilitators have to record these errors independently.

Scenario Completion Time

The time to complete each scenario, not including subjective evaluation durations, will be recorded.

Reporting Results

Reporting Results

Task 1: Bootstrap/ Import File(s)

Bootstrap/import file(s) Concern Recommendations
Critical Errors -- --
Non-Critical Errors

- Window to get bootstrap data blocked the “Bootstrap Data” page.

- @Homepage: give system name instead of “Welcome..”

- “Bootstrap” change it to: “import”

- Texts are not cleared after bootstrap is started

- 1% progress bar is not implemented

- Have everything in process view (e.g. Breadcrumb)

- While importing, disable all browse and bootstrap buttons.

Scenario Completion Time 4 minutes --

Task 2: Add Staff Costs

Add staff costs Concern Recommendations
Critical Errors -- --
Non-Critical Errors - Unsure if the default figures can be changed / edited. - Stick to current cost

- Add shortcut key like control S to simulate

Scenario Completion Time 10 seconds --

Task 3: Add Uncertainties

Add uncertainties Concern Recommendations
Critical Errors -- --
Non-Critical Errors - Flight delay rate should not be just MEAN and STDEV. They may not be reflective of certain infrequent scenario, e.g. strike that causes 3 hrs delay. Happens every year.

- Unsure if all stdev has to be filled. - What to fill for stdev?

- Allow users to add input on when (certain days and certain flight) will have xx hours of delay.

- Show ALL flight delay rate and allow user to change the data just like in Manage Airline Requirements.

Scenario Completion Time 10 seconds --

Task 4: Run Simulation

Run simulation Concern Recommendations
Critical Errors - Low staff utilisation rate --
Non-Critical Errors - Do not have print outs, they will slow down the system. - Kevin will get back to us on the staff utilization rate

- The “optimize schedule” button run from the homepage does not contain BACK button.

- Generate optimal schedule button is not obvious

- The circle does not display

- Does not display the remaining time required for the simulation to complete

- Staff Utilization Rate = Total Working Time within roster hour / Total Roster Hours AssigneD (this excludes the overtime hours and recall hours)

Scenario Completion Time 13 minutes (run simulation for 30 days)

3 minutes 10 seconds (run simulation for 7 days) 2 minutes 49 seconds (run simulation for 1 day)


Task 5: View staff schedule (in Gantt chart)

View staff schedule (in Gantt chart) Concern Recommendations
Critical Errors - Scheduling result is off. - Optimum number of people needed? (optional)

- Cannot delete list in the table. Need to fix this.

Non-Critical Errors - Cannot change the week

- The Gantt chart looks weird

- Try putting Flight Number at the blue bubble

Scenario Completion Time 2 minutes --

Task 6: Task 5: Add airline requirements

Add airline requirements Concern Recommendations
Critical Errors - - - -
Non-Critical Errors - Labels are cut, adjust properly

- Prompt overwrite

- Save button does not display well.

- Have textfields for HRS and MINS as unit measurement.

- Also cater to ‘transfer desk’ and ‘premier check-in lounge’ staff → nut they have the same attributes as normal counter departure staff. So, nothing needs to be changed. (check with client again)

- Clear the text fields after SAVE successfully.

Scenario Completion Time 4 minutes --

Task 7: Generate result

Generate Result Concern Recommendations
Critical Errors - - - Optimum number of people needed? (optional)

- Cannot delete list in the table. Need to fix this.

Non-Critical Errors - “Manage Result” name for button is misleading. User was looking for “Generate Result”

- OT hours: The actual data may be overvalued as staff tends to round up the each OT hour.

- Aware of licensing issue when using third party application (the Gantt chart?)

- Change the background of the calendar

Scenario Completion Time 20 seconds --

Subjective Evaluations

Subjective Evaluations

Subjective evaluations regarding ease of use and satisfaction will be collected via questionnaires. There are 2 participants, thus results from both participants will be combined or averaged whenever it is necessary.

Each of the two participants contributes 50% to their answers. Not all sections are answered, thus not all questions have a total of 100% weight.

Navigation Impression

Navigation Impression Agree Neutral Disagree
It is easy to find my way around the system -- -- 50%
It is easy to remember where to find things -- 50% --
The system is well-suited to first-time users -- -- 50%


-‘Back’ button at simulation is missing

- Have a flow. Not sure which button to select?

- Need more instructions for first time users.

Look and Feel

Look and Feel Agree Neutral Disagree
The interface design is simple 100% -- --
The size and layout of the application is optimal -- 100% --

Comment(s): NA


Functions Agree Neutral Disagree
Each function has a clear purpose 50% 50% --


- Must be more explicit on description

Bootstrap/import file(s)

Bootstrap/import file(s) Agree Disagree
The function works well 100% --
The function takes reasonable amount of time 100% --
The function provides right amount of information 100% --
The result/outcome of the function is right 100% --


- “Bootstrap” should be changed to “import”

- Disable ‘browse’ button when bootstrapping

Add staff costs [Manage Simulation Parameters]

Add staff costs [Manage Simulation Parameters] Agree Disagree
The function works well 100% --
The function takes reasonable amount of time 100% --
The function provides right amount of information 100% --
The result/outcome of the function is right 100% --

Comment(s): NA

Add uncertainties [Manage Sim. Parameters]

Add uncertainties [Manage Sim. Parameters] Agree Disagree
The function works well 100% --
The function takes reasonable amount of time 100% --
The function provides right amount of information 100% --
The result/outcome of the function is right 100% --


- Unit of measurement change to “hrs+mins”

- Allow shortcut key(e.g. [Alt + S] to start simulation)

Run Simulation

Run Simulation Agree Disagree
The function works well 50% --
The function takes reasonable amount of time 50% --
The function provides right amount of information 50% --
The result/outcome of the function is right 50% --


- Inconsistent textbox format

- Progress bar is not showing

- Exception handling. Encountered null pointer exception.

View staff schedule [in Gantt Chart]

View staff schedule [in Gantt Chart] Agree Disagree
The function works well -- 50%
The function takes reasonable amount of time 50% --
The function provides right amount of information 50% --
The result/outcome of the function is right -- 50%


- Staff schedule is incorrect

- Please add ‘flight number’ in Gantt chart

Add airline requirements

Add airline requirements Agree Disagree
The function works well 50% --
The function takes reasonable amount of time 50% --
The function provides right amount of information 50% --
The result/outcome of the function is right 50% --


- Success message not seen properly

- Can have guideline

Generate Result

Generate result Agree Disagree
The function works well 50% 50%
The function takes reasonable amount of time 100% --
The function provides right amount of information 100% --
The result/outcome of the function is right -- 50%


- Cannot delete PDF record

- How does this differ from “run simulation”?

Overall Impression

Overall Impression Agree Neutral Disagree
Satisfied with the system 100% -- --

Comment(s): NA

1. “What did you like best about this system?”

- Cost calculation is beneficial.

2. What did you like least about this system?

- NA

3. If you could make changes to this system, what change would you make?

- NA

4. Do you have any questions or comments about the system or your experiences with it?

- It could be a good tool

Reporting Conclusions

Reporting Conclusions

• Client was satisfied with the system. However, there are more to be improved in terms of the presentation and navigation of the application.

• Client, especially another participant gained better and clearer understanding on what the application delivers after testing the application.

• There are critical errors on the logic/formula for staff utilization rate and staff working hours that can be further improved to increase the accuracy of the calculation.

• Non-critical errors will also be solved.

Project Management

Time Management

Time Management



Iterations Parallel Plan


Schedule Metric



Schedule Tracking

Click here to see our Schedule Tracking.

Quality Management

Quality Management

Bug Metric



Bug Tracking

Click here to see our Bug Log.

Risk Management

Risk Management

Risk Metric


Risk Table

S/N Risk Statement Likelihood (H/M/L) Impact (H/M/L) Level (Derived) Mitigation Strategy &/or Contingency Plan
1 Managing expectation of new person in charge from client company

New client in charge may have different expectation of the software deliverable and functions.

H H A Prepare software introduction to client and other potential users. Earlier handover to client company.

Ensure that new person in charge is also clear of the software objective and current client is also available to help smoothen the handover process.

2 Limited real data to work with

Sample data provided might not be sufficient to represent the big picture.

H H A Consistent and thorough demonstration of system to client

Ensure that all doubts are clarified during client meeting. Bi-monthly demos are done to client to ensure correctness of our data. Try as far as possible to generate sensible data from provided samples.

3 Long simulation duration taken

Large amount of data to handle which will cause simulation duration to be long.


Explore ways to increase the efficiency of code (e.g. caching, threading) Also, the team will only consider necessasry data.

4 Low productivity

Unproductive team members esp when 3 out of 5 members are doing internship.

M M B Set sensible timeline and do progress check

Team has to set a sensible and achievable timeline. Also, project manager

5 Unexpected bugs

As the project is dealing with complex algorithm, unexpected bugs may surface and may not be very visible or consistently appearing.

M M B More thorough testing with more data test and more buffer time

Team has to ensure that each testing period in each iteration is well used to identify all possible bugs. This is also to prevent any old bugs from surfacing later in time. Team needs to assign more buffer time to prepare for the unexpected bugs.

6 Resource limitation

Final will be a great challenge as our PM will be away on the last week of IS480 course for personal reason.

M M B Prepare early and finish the big tasks early

Team can plan to finish the tasks, including administrative work early.

7 Sudden change of requirement

Client may have sudden change or addition of requirement.

L M C Regular communication and agreement sign off with client

Team will have frequent bi-monthly meeting with client to align the project requirement between both parties. Besides, agreement made between two parties helps to clarify the requirement.