HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T1 Bumblebee Project Documentation User Acceptance Test"

From IS480
Jump to navigation Jump to search
 
(83 intermediate revisions by 3 users not shown)
Line 24: Line 24:
 
|}
 
|}
  
==<div style="color: #ffffff; font-size:0%">Objectives</div>==
+
 
 +
===<div style="color: #ffffff; font-size:0%">User Testing 1, 17 September 2012</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Objectives</b>
+
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>User Testing 1, 17 September 2012</b>
 
|-
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
Line 33: Line 34:
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
  
The goals and objectives of usability testing:
+
'''The goals and objectives of usability testing:'''
  
 
• Record and document general feedback and first impressions
 
• Record and document general feedback and first impressions
  
• Identify any potential concerns to address regarding application usability, presentation, and navigation.
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
 +
 
 +
• Get feedback on the usefulness and accuracy of the functions developed
 +
 
 +
• To match client expectations on the system developed
 +
 
  
• Get feedback on the usefulness and accuracy of the functions developed.
+
'''User Testing Conclusion:'''
  
To match client expectations on the system developed.
+
Client was satisfied with the system. However, there are more to be improved in terms of the presentation and navigation of the application
 +
 
 +
• Client, especially another participant gained better and clearer understanding on what the application delivers after testing the application
 +
 
 +
• There are critical errors on the logic/formula for staff utilization rate and staff working hours that can be further improved to increase the accuracy of the calculation
 +
 
 +
• Non-critical errors will also be solved
 +
 
 +
 
 +
Click [[Media:BumblebeeUserTesting1Report.pdf‎|here]] to download our report
  
 
|-
 
|-
Line 49: Line 64:
 
|-
 
|-
 
|}
 
|}
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 
  
==<div style="color: #ffffff; font-size:0%">Methodology</div>==
+
===<div style="color: #ffffff; font-size:0%">User Testing 2, 25 October 2012</div>===
 
{| class="wikitable" cellpadding="15"
 
{| class="wikitable" cellpadding="15"
 
|-
 
|-
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Methodology</b>
+
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>User Testing 2, 25 October 2012</b>
 
|-
 
|-
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
|-
 
|-
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
===User Testing Environment===
 
  
Computer platform : Intel Pentium Processor
+
'''The goals and objectives of usability testing:'''
  
Screen resolution : 1028 x 768
+
• Record and document general feedback and first impressions
  
Operating System : Windows XP
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
  
Set-up required : Computer date format (English (Australia)) of d/MM/YYYY
+
• Get feedback on the usefulness and accuracy of the functions developed
  
===Participants===
+
• To match client expectations on the system developed
  
The participants will attempt to complete a set of scenarios presented to them and to provide feedback regarding the usability and acceptability of the application.
 
  
1. Kevin Choy, SATS Airline Relations Manager - Person-in-charge for this project.
+
'''User Testing Conclusion:'''
  
2. Goh Wei Xuan, SATS Airline Relations Manager
+
• Client was satisfied with the system
  
===Procedure===
+
• Current result seems close and better than the actual result under manual scheduling. However, close monitor of result with client is highly needed to ensure that the result is indeed valid
  
====Instructions====
+
• Team will consistently send the latest file to client for independent testing
  
c
 
  
1. Each user will be accompanied by 1 facilitator.
+
Click [[Media:BumblebeeUserTesting2Report.pdf‎|here]] to download our report
  
2. Users are encouraged to verbalize their movements, purpose, and problems.
+
|-
  
3. Facilitators will record mistakes and questions made by users during testing.
+
|-
 +
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
|-
 +
|}
  
4. To start the test, click on the file named “START.bat” found in folder named “SATS_Bumblebee_Beta_v5”.
+
===<div style="color: #ffffff; font-size:0%">Heuristic Evaluation, 12-15 November 2012</div>===
 +
{| class="wikitable" cellpadding="15"
 +
|-
 +
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>Heuristic Evaluation, 12-15 November 2012</b>
 +
|-
 +
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
|-
 +
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
  
5. All sample files needed for testing are found in: SATS_Bumblebee_Beta_v5/data
+
'''The goals and objectives of usability testing:'''
  
6. Database used to store imported data is also found in ROOT folder.
+
• Record and document general feedback and first impressions
  
7. Users are allowed to change their input(s) to verify data validity.
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
  
8. Users are to complete the tasks stated below. After completing each task, users have to answer the test questions pertaining to the specific task.
 
  
====Tasks====
+
'''Heuristic Evaluation Conclusion:'''
  
These are the task descriptions given to clients:
+
• Users have no problem in navigating around the system in general
  
Below are tasks for users to complete.
+
• There is a need for more instructions from pages to pages
  
1. Bootstrap/import files(s)
 
  
This task is for user to import data from excel files such as Flight Schedule Departure, Flight Schedule Arrival, Staff Records, etc. into the application. The application will then use these data for simulation purpose in the later step.
+
Click [[Media:BumblebeeHeuristicEvaluationReport.pdf‎|here]] to download our report
  
2. Add staff costs
+
|-
  
This task is to record various costs in hiring staff into the application.
+
|-
 +
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
|-
 +
|}
  
3. Add uncertainties
+
===<div style="color: #ffffff; font-size:0%">User Testing 3, 22 November 2012</div>===
 +
{| class="wikitable" cellpadding="15"
 +
|-
 +
| style="background:#ffffff;; color: #000000; font-weight: bold; text-indent: 2px; width: 32%; font-size:22px; text-align:lrft; border: 1px solid #ffffff" valign="top" | <b>User Testing 3, 22 November 2012</b>
 +
|-
 +
| style= "background:#000000; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
 +
|-
 +
| style= "background:#FBB917; color: #000000; padding: 10px 4px 0 15px; font-size: 13px;" valign="top"  |
  
This task is to record the mean and standard deviation of different uncertainties that will affect the initial schedule prepared by the application. Simulation period = 7 days (represents the number of days the data is to be generate).
+
'''The goals and objectives of usability testing:'''
  
4. Run simulation
+
• Record and document general feedback and first impressions
 
 
Run simulation to start assigning staff to different job assignments.
 
 
 
5. View staff schedule (in Gantt Chart)
 
 
 
This allows user to view and compare between a staff’s planned and actual working time.
 
 
 
6. Add airline requirements
 
 
 
Airlines have several different requirements on number of CSA and CSO needed.
 
 
 
This task is to record the individual requirements into the database. The input data will be used for simulation purpose in the later step.
 
 
 
7. Generate result
 
 
 
This task is to view the result generated in PDF format.
 
 
 
===Team Roles===
 
 
 
Overall in-charge (Yosin Anggusti)
 
  
- Provide training overview prior to usability testing
+
• Identify any potential concerns to address regarding application usability, presentation, and navigation
  
- Defines usability and purpose of usability testing to participants
+
• Get feedback on the usefulness and accuracy of the functions developed
  
Facilitators (Glorya Marie, Suriyanti)
+
• To match client expectations on the system developed
  
- Evaluate on the application and user interaction with the application, rather than evaluating on the user
 
  
- Facilitator will observe and enter user behavior and user comments.
+
'''User Testing Conclusion:'''
  
- Responds to participant’s requests for assistance
+
• Client was satisfied with the system.
  
Test Observers (Yosin Anggusti)
+
• Results are reasonable
  
- Silent observers
+
• A video as user manual will be done to aid user in using the software
  
- Assists the data logger in identifying problems, concerns, coding bugs, and procedural errors
 
  
  
 +
Click [[Media:BumblebeeUserTesting2Report.pdf‎|here]] to download our report
 
|-
 
|-
  
Line 164: Line 172:
 
|-
 
|-
 
|}
 
|}
<div style = "text-align:right;"><font ="color:#000000; size:"12px;"><u>[[IS480_Team_wiki:_2012T1_Bumblebee_Project_Documentation_User_Acceptance_Test#Top|Back to Top]]</u></font></div>
 

Latest revision as of 11:57, 4 December 2012

Bao.png
Home Project Overview Project Documentation Project Management Learning and Growth
Diagrams User Interface User Manual User Testing Meeting Minutes Presentations


User Testing 1, 17 September 2012

User Testing 1, 17 September 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed


User Testing Conclusion:

• Client was satisfied with the system. However, there are more to be improved in terms of the presentation and navigation of the application

• Client, especially another participant gained better and clearer understanding on what the application delivers after testing the application

• There are critical errors on the logic/formula for staff utilization rate and staff working hours that can be further improved to increase the accuracy of the calculation

• Non-critical errors will also be solved


Click here to download our report

User Testing 2, 25 October 2012

User Testing 2, 25 October 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed


User Testing Conclusion:

• Client was satisfied with the system

• Current result seems close and better than the actual result under manual scheduling. However, close monitor of result with client is highly needed to ensure that the result is indeed valid

• Team will consistently send the latest file to client for independent testing


Click here to download our report

Heuristic Evaluation, 12-15 November 2012

Heuristic Evaluation, 12-15 November 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation


Heuristic Evaluation Conclusion:

• Users have no problem in navigating around the system in general

• There is a need for more instructions from pages to pages


Click here to download our report

User Testing 3, 22 November 2012

User Testing 3, 22 November 2012

The goals and objectives of usability testing:

• Record and document general feedback and first impressions

• Identify any potential concerns to address regarding application usability, presentation, and navigation

• Get feedback on the usefulness and accuracy of the functions developed

• To match client expectations on the system developed


User Testing Conclusion:

• Client was satisfied with the system.

• Results are reasonable

• A video as user manual will be done to aid user in using the software


Click here to download our report