HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T1 Bumblebee Project Documentation User Acceptance Test"

From IS480
Jump to navigation Jump to search
 
(26 intermediate revisions by 2 users not shown)
Line 25: Line 25:
  
  
 
+
===<div style="color: #ffffff; font-size:0%">User Testing 1, 17 September 2012</div>===
 
 
 
 
 
 
 
 
==<div style="color: #ffffff; font-size:0%">Objectives</div>==
 
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b><u>User Testing 1 (17 September 2012)</u>
+
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>User Testing 1, 17 September 2012</b>
Objectives</b>
 
 
|-
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
Line 40: Line 34:
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
  
The goals and objectives of usability testing:
+
'''The goals and objectives of usability testing:'''
  
 
• Record and document general feedback and first impressions
 
• Record and document general feedback and first impressions
  
• Identify any potential concerns to address regarding application usability, presentation, and navigation.
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
  
• Get feedback on the usefulness and accuracy of the functions developed.
+
• Get feedback on the usefulness and accuracy of the functions developed
  
• To match client expectations on the system developed.
+
• To match client expectations on the system developed
 
 
|-
 
 
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
|}
 
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
  
==<div style="color: #ffffff; font-size:0%">Methodology</div>==
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Methodology</b>
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
===User Testing Environment===
 
  
Computer platform : Intel Pentium Processor
+
'''User Testing Conclusion:'''
  
Screen resolution : 1028 x 768
+
• Client was satisfied with the system. However, there are more to be improved in terms of the presentation and navigation of the application
  
Operating System : Windows XP
+
• Client, especially another participant gained better and clearer understanding on what the application delivers after testing the application
  
Set-up required : Computer date format (English (Australia)) of d/MM/YYYY
+
• There are critical errors on the logic/formula for staff utilization rate and staff working hours that can be further improved to increase the accuracy of the calculation
  
===Participants===
+
• Non-critical errors will also be solved
  
The participants will attempt to complete a set of scenarios presented to them and to provide feedback regarding the usability and acceptability of the application.
 
 
1. Kevin Choy, SATS Airline Relations Manager - Person-in-charge for this project.
 
 
2. Goh Wei Xuan, SATS Airline Relations Manager
 
 
===Procedure===
 
 
====Instructions====
 
 
These instructions were given to our clients:
 
 
1. Each user will be accompanied by 1 facilitator.
 
 
2. Users are encouraged to verbalize their movements, purpose, and problems.
 
 
3. Facilitators will record mistakes and questions made by users during testing.
 
 
4. To start the test, click on the file named “START.bat” found in folder named “SATS_Bumblebee_Beta_v5”.
 
 
5. All sample files needed for testing are found in:  SATS_Bumblebee_Beta_v5/data
 
 
6. Database used to store imported data is also found in ROOT folder.
 
 
7. Users are allowed to change their input(s) to verify data validity.
 
 
8. Users are to complete the tasks stated below. After completing each task, users have to answer the test questions pertaining to the specific task.
 
 
====Tasks====
 
 
These are the task descriptions given to clients:
 
 
Below are tasks for users to complete.
 
 
1. Bootstrap/import files(s)
 
 
This task is for user to import data from excel files such as Flight Schedule Departure, Flight Schedule Arrival, Staff Records, etc. into the application. The application will then use these data for simulation purpose in the later step.
 
 
2. Add staff costs
 
 
This task is to record various costs in hiring staff into the application.
 
 
3. Add uncertainties
 
 
This task is to record the mean and standard deviation of different uncertainties that will affect the initial schedule prepared by the application. Simulation period = 7 days (represents the number of days the data is to be generate).
 
 
4. Run simulation
 
 
Run simulation to start assigning staff to different job assignments.
 
 
5. View staff schedule (in Gantt Chart)
 
 
This allows user to view and compare between a staff’s planned and actual working time.
 
 
6. Add airline requirements
 
 
Airlines have several different requirements on number of CSA and CSO needed.
 
 
This task is to record the individual requirements into the database. The input data will be used for simulation purpose in the later step.
 
 
7. Generate result
 
 
This task is to view the result generated in PDF format.
 
 
===Team Roles===
 
 
Overall in-charge (Yosin Anggusti)
 
 
- Provide training overview prior to usability testing
 
 
- Defines usability and purpose of usability testing to participants
 
 
Facilitators (Glorya Marie, Suriyanti)
 
 
- Evaluate on the application and user interaction with the application, rather than evaluating on the user
 
 
- Facilitator will observe and enter user behavior and user comments.
 
 
- Responds to participant’s requests for assistance
 
 
Test Observers (Yosin Anggusti)
 
 
- Silent observers
 
 
- Assists the data logger in identifying problems, concerns, coding bugs, and procedural errors
 
  
 +
Click [[Media:BumblebeeUserTesting1Report.pdf‎|here]] to download our report
  
 
|-
 
|-
Line 171: Line 64:
 
|-
 
|-
 
|}
 
|}
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
  
==<div style="color: #ffffff; font-size:0%">Reporting Results</div>==
+
===<div style="color: #ffffff; font-size:0%">User Testing 2, 25 October 2012</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Usability Metrics</b>
+
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>User Testing 2, 25 October 2012</b>
 
|-
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
Line 182: Line 74:
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
  
===Critical Errors===
+
'''The goals and objectives of usability testing:'''
Critical errors are deviations of results from the actual result. These errors will cause the task to fail. Facilitators are to records there critical errors.
 
  
===Non-Critical Errors===
+
• Record and document general feedback and first impressions
Non-critical errors are usually procedural, in which the participant does not complete a task in the most optimal means (e.g. excessive steps, initially selecting the wrong function, attempting to edit an un-editable field).
 
These errors may not be detected by the user himself. Facilitators have to record these errors independently.
 
 
 
===Scenario Completion Time===
 
The time to complete each scenario, not including subjective evaluation durations, will be recorded.
 
 
 
 
 
|-
 
 
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
|}
 
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
  
==<div style="color: #ffffff; font-size:0%">Reporting Results</div>==
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
{| class="wikitable" cellpadding="15"
 
|-
 
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Reporting Results</b>
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
  
===Task 1: Bootstrap/ Import File(s)===
+
• Get feedback on the usefulness and accuracy of the functions developed
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Bootstrap/import file(s)</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Recommendations</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  |
 
- Window to get bootstrap data blocked the “Bootstrap Data” page.
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  |
 
- @Homepage: give system name instead of “Welcome..”
 
  
- “Bootstrap” change it to: “import”
+
• To match client expectations on the system developed
  
- Texts are not cleared after bootstrap is started
 
  
- 1% progress bar is not implemented
+
'''User Testing Conclusion:'''
  
- Have everything in process view (e.g. Breadcrumb)
+
• Client was satisfied with the system
  
- While importing, disable all browse  and bootstrap buttons.
+
• Current result seems close and better than the actual result under manual scheduling. However, close monitor of result with client is highly needed to ensure that the result is indeed valid
  
|-
+
• Team will consistently send the latest file to client for independent testing
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 4 minutes
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
  
===Task 2: Add Staff Costs===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Add staff costs</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Recommendations</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Unsure if the default figures can be changed / edited.
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Stick to current cost
 
- Add shortcut key like control S to simulate
 
  
 +
Click [[Media:BumblebeeUserTesting2Report.pdf‎|here]] to download our report
  
 
|-
 
|-
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 10 seconds
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
  
===Task 3: Add Uncertainties===
 
{| class="wikitable" cellpadding="15"
 
 
|-
 
|-
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Add uncertainties</b>
+
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Recommendations</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Flight delay rate should not be just MEAN and STDEV. They may not be reflective of certain infrequent scenario, e.g. strike that causes 3 hrs delay. Happens every year.
 
- Unsure if all stdev has to be filled.
 
- What to fill for stdev?
 
 
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Allow users to add input on when (certain days and certain flight) will have xx hours of delay.
 
- Show ALL flight delay rate and allow user to change the data just like in Manage Airline Requirements.
 
 
 
 
 
 
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 10 seconds
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
 
|-
 
|-
 
|}
 
|}
  
===Task 4: Run Simulation===
+
===<div style="color: #ffffff; font-size:0%">Heuristic Evaluation, 12-15 November 2012</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Run simulation</b>
+
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Heuristic Evaluation, 12-15 November 2012</b>
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top" | <b>Recommendations</b>
 
 
|-
 
|-
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
+
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Low staff utilisation rate
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
 
|-
 
|-
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
+
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Do not have print outs, they will slow down the system.
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Kevin will get back to us on the staff utilization rate
 
  
- The “optimize schedule” button run from the homepage does not contain BACK button.
+
'''The goals and objectives of usability testing:'''
  
- Generate optimal schedule button is not obvious
+
• Record and document general feedback and first impressions
  
- The circle does not display
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
  
- Does not display the remaining time required for the simulation to complete
 
  
- Staff Utilization Rate = Total Working Time within roster hour / Total Roster Hours AssigneD
+
'''Heuristic Evaluation Conclusion:'''
(this excludes the overtime hours and recall hours)
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 13 minutes (run simulation for 30 days)
 
3 minutes 10 seconds (run simulation for 7 days)
 
2 minutes 49 seconds (run simulation for 1 day)
 
  
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
+
• Users have no problem in navigating around the system in general
|-
 
|}
 
  
 +
• There is a need for more instructions from pages to pages
  
  
===Task 5: View staff schedule (in Gantt chart)===
+
Click [[Media:BumblebeeHeuristicEvaluationReport.pdf‎|here]] to download our report
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>View staff schedule (in Gantt chart)</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Recommendations</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Scheduling result is off.
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Optimum number of people needed? (optional)
 
- Cannot delete list in the table. Need to fix this.
 
 
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Cannot change the week
 
- The Gantt chart looks weird
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  |
 
 
 
- Try putting Flight Number at the blue bubble
 
 
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 2 minutes
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
 
===Task 6: Task 5: Add airline requirements===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Add airline requirements</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Recommendations</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - -
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - -
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Labels are cut, adjust properly
 
 
 
- Prompt overwrite
 
 
 
- Save button does not display well.
 
 
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  |
 
 
 
- Have textfields for HRS and MINS as unit measurement.
 
 
 
- Also cater to ‘transfer desk’ and ‘premier check-in lounge’ staff → nut they have the same attributes as normal counter departure staff. So, nothing needs to be changed.
 
(check with client again)
 
 
 
- Clear the text fields after SAVE successfully.
 
  
 
|-
 
|-
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 4 minutes
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
===Task 7: Generate result===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Generate Result</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Concern</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | <b>Recommendations</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - -
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - Optimum number of people needed? (optional)
 
- Cannot delete list in the table. Need to fix this.
 
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Non-Critical Errors
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | - “Manage Result” name for button is misleading. User was looking for “Generate Result”
 
- OT hours: The actual data may be overvalued as staff tends to round up the each OT hour.
 
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  |
 
 
- Aware of licensing issue when using third party application (the Gantt chart?)
 
 
- Change the background of the calendar
 
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 30%; font-size: 13px;" valign="top"  | Scenario Completion Time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | 20 seconds
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 30%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
|-
 
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
|}
 
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
 
  
 
|-
 
|-
Line 437: Line 134:
 
|-
 
|-
 
|}
 
|}
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
  
==<div style="color: #ffffff; font-size:0%">Subjective Evaluations</div>==
+
===<div style="color: #ffffff; font-size:0%">User Testing 3, 22 November 2012</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Subjective Evaluations</b>
+
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>User Testing 3, 22 November 2012</b>
 
|-
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
|-
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
Subjective evaluations regarding ease of use and satisfaction will be collected via questionnaires. There are 2 participants, thus results from both participants will be combined or averaged whenever it is necessary.
 
  
Each of the two participants contributes 50% to their answers. Not all sections are answered, thus not all questions have a total of 100% weight.
+
'''The goals and objectives of usability testing:'''
  
===Navigation Impression===
+
• Record and document general feedback and first impressions
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Navigation Impression</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Neutral</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | It is easy to find my way around the system
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | It is easy to remember where to find things
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The system is well-suited to first-time users
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
|-
 
|}
 
Comment(s):
 
  
-‘Back’ button at simulation is missing
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
  
- Have a flow. Not sure which button to select?
+
• Get feedback on the usefulness and accuracy of the functions developed
  
- Need more instructions for first time users.
+
• To match client expectations on the system developed
  
===Look and Feel===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Look and Feel</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Neutral</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The interface design is simple
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The size and layout of the application is optimal
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
  
Comment(s):  NA
+
'''User Testing Conclusion:'''
===Functions===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Functions  </b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Neutral</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | Each function has a clear purpose
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
 
Comment(s):
 
  
- Must be more explicit on description
+
• Client was satisfied with the system.
===Bootstrap/import file(s)===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Bootstrap/import file(s)</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
 
Comment(s):
 
 
 
- “Bootstrap” should be changed to “import”
 
 
 
- Disable ‘browse’ button when bootstrapping
 
 
 
===Add staff costs [Manage Simulation Parameters]===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Add staff costs [Manage Simulation Parameters]</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
 
Comment(s):  NA
 
===Add uncertainties [Manage Sim. Parameters]===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Add uncertainties [Manage Sim. Parameters]</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
Comment(s):
 
 
 
- Unit of measurement change to “hrs+mins”
 
 
 
- Allow shortcut key(e.g. [Alt + S] to start simulation)
 
 
 
===Run Simulation===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Run Simulation</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
 
Comment(s):
 
 
 
- Inconsistent textbox format
 
 
 
- Progress bar is not showing
 
 
 
- Exception handling. Encountered null pointer exception.
 
 
 
===View staff schedule [in Gantt Chart]===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>View staff schedule [in Gantt Chart]</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
|-
 
|}
 
Comment(s):
 
 
 
- Staff schedule is incorrect
 
 
 
- Please add ‘flight number’ in Gantt chart
 
 
 
===Add airline requirements===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Add airline requirements</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
 
 
Comment(s):
 
 
 
- Success message not seen properly
 
 
 
- Can have guideline
 
===Generate Result===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Generate result</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |The function works well
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function takes reasonable amount of time
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The function provides right amount of information
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | The result/outcome of the function is right
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 50%
 
|-
 
|}
 
 
 
Comment(s):
 
 
 
- Cannot delete PDF record
 
 
 
- How does this differ from “run simulation”?
 
 
 
===Overall Impression===
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  | <b>Overall Impression</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Agree</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Neutral</b>
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 25%; font-size: 13px;" valign="top"  | <b>Disagree</b>
 
|-
 
| style= "background:#000000; color: #ffffff; padding: 10px 4px 0 15px; width: 50%; font-size: 13px;" valign="top"  |Satisfied with the system
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | 100%
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
| style= "background:#ffffff; color: #000000; padding: 10px 4px 0 15px; width: 25%;font-size: 13px;" valign="top"  | --
 
|-
 
|}
 
Comment(s):  NA
 
 
 
1. “What did you like best about this system?”
 
 
 
- Cost calculation is beneficial.
 
 
 
2. What did you like least about this system?
 
 
 
- NA
 
 
 
3. If you could make changes to this system, what change would you make?
 
 
 
- NA
 
 
 
4. Do you have any questions or comments about the system or your experiences with it?
 
 
 
- It could be a good tool
 
 
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
|}
 
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
 
 
==<div style="color: #ffffff; font-size:0%">Reporting Conclusions</div>==
 
{| class="wikitable" cellpadding="15"
 
|-
 
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Reporting Conclusions</b>
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
  
Client was satisfied with the system. However, there are more to be improved in terms of the presentation and navigation of the application.
+
Results are reasonable
  
Client, especially another participant gained better and clearer understanding on what the application delivers after testing the application.
+
A video as user manual will be done to aid user in using the software
  
• There are critical errors on the logic/formula for staff utilization rate and staff working hours that can be further improved to increase the accuracy of the calculation.
 
  
• Non-critical errors will also be solved.
 
  
 +
Click [[Media:BumblebeeUserTesting2Report.pdf‎|here]] to download our report
 
|-
 
|-
  
Line 797: Line 172:
 
|-
 
|-
 
|}
 
|}
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 

Latest revision as of 11:57, 4 December 2012

Bao.png
Home Project Overview Project Documentation Project Management Learning and Growth
Diagrams User Interface User Manual User Testing Meeting Minutes Presentations


User Testing 1, 17 September 2012

User Testing 1, 17 September 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed


User Testing Conclusion:

• Client was satisfied with the system. However, there are more to be improved in terms of the presentation and navigation of the application

• Client, especially another participant gained better and clearer understanding on what the application delivers after testing the application

• There are critical errors on the logic/formula for staff utilization rate and staff working hours that can be further improved to increase the accuracy of the calculation

• Non-critical errors will also be solved


Click here to download our report

User Testing 2, 25 October 2012

User Testing 2, 25 October 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed


User Testing Conclusion:

• Client was satisfied with the system

• Current result seems close and better than the actual result under manual scheduling. However, close monitor of result with client is highly needed to ensure that the result is indeed valid

• Team will consistently send the latest file to client for independent testing


Click here to download our report

Heuristic Evaluation, 12-15 November 2012

Heuristic Evaluation, 12-15 November 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation


Heuristic Evaluation Conclusion:

• Users have no problem in navigating around the system in general

• There is a need for more instructions from pages to pages


Click here to download our report

User Testing 3, 22 November 2012

User Testing 3, 22 November 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed


User Testing Conclusion:

• Client was satisfied with the system.

• Results are reasonable

• A video as user manual will be done to aid user in using the software


Click here to download our report