Difference between revisions of "IS480 Team wiki: 2013T1 ThunderBolt Project Documentation Usability Test"
(17 intermediate revisions by the same user not shown) | |||
Line 48: | Line 48: | ||
</td> | </td> | ||
</tr></table> | </tr></table> | ||
+ | ===<div style="background: #d5a10d; padding: 15px; font-weight: bold; line-height: 0.3em"><font color= #FFFFFF>Internal Testing</font></div>=== | ||
+ | Internal testing was conducted using regression method at end of each iteration. Refer to our test plan, test case and results below.<br> | ||
+ | '''Click [https://www.dropbox.com/s/7o3heujyzj23na2/test_plan.docx here] for our test plan'''<br> | ||
+ | '''Click [https://www.dropbox.com/s/f6prulqjh3n3f6p/Test%20cases%20and%20results.xlsx here] for our test cases by iterations'''<br> | ||
+ | |||
===<div style="background: #d5a10d; padding: 15px; font-weight: bold; line-height: 0.3em"><font color= #FFFFFF>Usability Test 1</font></div>=== | ===<div style="background: #d5a10d; padding: 15px; font-weight: bold; line-height: 0.3em"><font color= #FFFFFF>Usability Test 1</font></div>=== | ||
====<font color="#200772">Date and Venue</font>==== | ====<font color="#200772">Date and Venue</font>==== | ||
Line 425: | Line 430: | ||
|} | |} | ||
<br> | <br> | ||
− | User Testing 2 has been expanded from User Testing 1 to include 5 more features, which are highlighted in yellow. | + | '''User Testing 2 has been expanded from User Testing 1 to include 5 more features, which are highlighted in yellow.''' |
====<font color="#200772">Testing Method</font>==== | ====<font color="#200772">Testing Method</font>==== | ||
Line 435: | Line 440: | ||
|} | |} | ||
====<font color="#200772">Test Results</font>==== | ====<font color="#200772">Test Results</font>==== | ||
+ | '''Comparison of User Testing 1 survey result and User Testing 2 survey result. Overall, MORE users agree with our system!<br>''' | ||
{| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | {| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | ||
|scope="row" width="1000" style="text-align: left; background: #eeeeee"| | |scope="row" width="1000" style="text-align: left; background: #eeeeee"| | ||
Line 468: | Line 474: | ||
:4. '''Participant Informed Consent Form'''<br> | :4. '''Participant Informed Consent Form'''<br> | ||
:#Consent Form: [https://www.dropbox.com/s/6ca0aaoew2n2zcs/Participant%20Informed%20Constent%20Form.pdf] | :#Consent Form: [https://www.dropbox.com/s/6ca0aaoew2n2zcs/Participant%20Informed%20Constent%20Form.pdf] | ||
+ | |} | ||
+ | |||
+ | ===<div style="background: #d5a10d; padding: 15px; font-weight: bold; line-height: 0.3em"><font color= #FFFFFF>Usability Test 3</font></div>=== | ||
+ | ====<font color="#200772">Date and Venue</font>==== | ||
+ | {| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | ||
+ | |scope="row" width="1000" style="text-align: left; background: #eeeeee"| | ||
+ | '''Date: 18 Nov 2013'''<br> | ||
+ | '''Time: 2pm - 7pm'''<br> | ||
+ | '''Venue: SIS MR4.3'''<br> | ||
+ | |} | ||
+ | |||
+ | ====<font color="#200772">Participants</font>==== | ||
+ | {| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | ||
+ | |scope="row" width="1000" style="text-align: left; background: #eeeeee"| | ||
+ | '''No. of Participants: 25'''<br> | ||
+ | '''Participant Profile:'''<br> | ||
+ | - Admin<br> | ||
+ | - 4 Supervisor/Reviewer<br> | ||
+ | - 3 TA<br> | ||
+ | - 17 Students<br> | ||
+ | [[Image:UT1-3ParticipantProfile.png|530px]]<br><br> | ||
+ | |} | ||
+ | ====<font color="#200772">Scope of Test</font>==== | ||
+ | {| class="wikitable" style="text-align: center; height:50px" | ||
+ | |+ | ||
+ | |- | ||
+ | ! scope="col" width="150" style="background-color:#200772"| <font color="#ffffff">Function</font> | ||
+ | ! scope="col" width="50" style="background-color:#200772"| <font color="#ffffff">Admin</font> | ||
+ | ! scope="col" width="130" style="background-color:#200772"| <font color="#ffffff">Course Coordinator</font> | ||
+ | ! scope="col" width="50" style="background-color:#200772"| <font color="#ffffff">Supervisor/Reviewer</font> | ||
+ | ! scope="col" width="50" style="background-color:#200772"| <font color="#ffffff">TA</font> | ||
+ | ! scope="col" width="50" style="background-color:#200772"| <font color="#ffffff">Student</font> | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Single Sign On | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | ! scope="row" style="background-color:#eeeeee"|Role switching | ||
+ | |style="text-align: center;|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Create schedule | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|View schedule | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Edit schedule | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Archive schedule | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Create booking | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Add optional attendees | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|View booking | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Edit booking | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Delete booking | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Approve/Reject booking | ||
+ | |style="text-align: center;|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Reject booking with reason | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|Cancel booking | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|View my bookings | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|Presentation subscription | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |- | ||
+ | ! scope="row" style="background-color:#eeeeee"|Supervisor Availability | ||
+ | |style="text-align: center;|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|TA sign up for filming | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|TA swap slots | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Add default milestones | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Add required attendees for different milestones | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#eeeeee"|Set presentation duration | ||
+ | |style="text-align: center;|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |style="text-align: center;"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|CSV file upload | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|Manage users | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|Generate reports | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|Admin/Faculty Dashboard | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |- | ||
+ | |||
+ | ! scope="row" style="background-color:#FBF8C7"|TA sign up dashboard | ||
+ | |style="text-align: center;background-color:#FBF8C7"|[[Image:transparent-green-checkmark.png|24px]] | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |style="text-align: center;background-color:#FBF8C7"|- | ||
+ | |- | ||
+ | |} | ||
+ | <br> | ||
+ | |||
+ | '''User Testing 3 has been expanded from User Testing 2 to include 8 more features, which are highlighted in yellow.''' | ||
+ | |||
+ | ====<font color="#200772">Testing Method</font>==== | ||
+ | {| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | ||
+ | |scope="row" width="1000" style="text-align: left; background: #eeeeee"| | ||
+ | A list of tasks based on different scenarios were given to participants and they need to follow the scenarios and figure out how to complete them in the system. | ||
+ | Respective facilitators who sat beside them need to observe and record down how they were using the system to complete the task assigned.<br> | ||
+ | Upon completion of all tasks, participants were given a link to complete a questionnaire with regards to the quality of emails they received from the system. | ||
+ | |} | ||
+ | |||
+ | ====<font color="#200772">Test Results</font>==== | ||
+ | {| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | ||
+ | |scope="row" width="1000" style="text-align: left; background: #eeeeee"| | ||
+ | |||
+ | '''Critical issues discovered during User Testing 3'''<br> | ||
+ | - Unable to handle conflicting concurrent actions. This happened when admin was trying to remove a faculty from the system while she was perform tasks in the system<br> | ||
+ | - Email ID mismatch. One of the faculty couldn't login to the system due to multiple different SSO IDs he has for different systems<br> | ||
+ | - Supervisors want their time slots to be pre-populate as unavailable instead of available<br> | ||
+ | - Some still complain that number of emails received were too much. The email on adding an optional attendee can be reduced<br> | ||
+ | [[Image:importanceofemail.PNG|400px]][[Image:languageofemail.PNG|400px]]<br> | ||
+ | |} | ||
+ | |||
+ | ====<font color="#200772">Documentation</font>==== | ||
+ | {| style="border: 1px solid darkgray; text-align: center;font-siz:14px; height:50px" | ||
+ | |scope="row" width="330" style="text-align: left; background: #eeeeee"| | ||
+ | :1. '''Test Plan''' | ||
+ | :#Test plan: [https://www.dropbox.com/s/oqw1gz0l05ikbxu/Usability%20Test%203%20Plan.pdf]<br> | ||
+ | :2. '''Scenario Task Lists''' | ||
+ | :#Supervisor/Reviewer task: [https://www.dropbox.com/s/nektjwquzloblyp/Task%20List%20for%20Supervisor.pdf]<br> | ||
+ | :#Student task: [https://www.dropbox.com/s/qxlkc8wmv90c6zr/Task%20List%20for%20Student.pdf]<br> | ||
+ | :#TA task: [https://www.dropbox.com/s/21eg48tpyx9x6t7/Task%20List%20for%20TA.pdf]<br> | ||
+ | :3. '''Qualtrics Survey Questions'''<br> | ||
+ | :#Survey Link: [http://smusg.asia.qualtrics.com/SE/?SID=SV_2fOb3G7Mh4ieRZr]<br> | ||
+ | :#Survey result: [https://www.dropbox.com/s/fn0ohswd5v4bd42/Survey%20Key%20Feedback.xlsx]<br> | ||
+ | :4. '''Participant Informed Consent Form'''<br> | ||
+ | :#Consent Form: [https://www.dropbox.com/s/imfibqb9moy7vtm/Participant%20Informed%20Constent%20Form.pdf] | ||
|} | |} |
Latest revision as of 03:15, 26 November 2013
About Us | Project Overview | Project Management | Project Documentation |
Diagrams | Prototype | Usability Test | Meeting Minutes | Downloads |
Internal Testing
Internal testing was conducted using regression method at end of each iteration. Refer to our test plan, test case and results below.
Click here for our test plan
Click here for our test cases by iterations
Usability Test 1
Date and Venue
Date: 2 Sep 2013 |
Participants
No. of Participants: 10 |
Scope of Test
Testing Method
A list of tasks based on different scenarios were given to participants and they need to follow the scenarios and figure out how to complete them in the system.
Respective facilitators who sat beside them need to observe and record down how they were using the system to complete the task assigned. |
Test Results
When asked about what do they like about our system: What asked what they dislike about the system: Improvements they would like to have to have a better user experience: |
Documentation
Usability Test 2
Date and Venue
Date: 4 Oct 2013 |
Participants
No. of Participants: 21 |
Scope of Test
User Testing 2 has been expanded from User Testing 1 to include 5 more features, which are highlighted in yellow.
Testing Method
A list of tasks based on different scenarios were given to participants and they need to follow the scenarios and figure out how to complete them in the system.
Respective facilitators who sat beside them need to observe and record down how they were using the system to complete the task assigned. |
Test Results
Comparison of User Testing 1 survey result and User Testing 2 survey result. Overall, MORE users agree with our system!
|
Documentation
Usability Test 3
Date and Venue
Date: 18 Nov 2013 |
Participants
No. of Participants: 25 |
Scope of Test
User Testing 3 has been expanded from User Testing 2 to include 8 more features, which are highlighted in yellow.
Testing Method
A list of tasks based on different scenarios were given to participants and they need to follow the scenarios and figure out how to complete them in the system.
Respective facilitators who sat beside them need to observe and record down how they were using the system to complete the task assigned. |