Difference between revisions of "IS480 Team wiki: 2012T1 Timber Werkz UsabilityTest"
(26 intermediate revisions by 2 users not shown) | |||
Line 64: | Line 64: | ||
{| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | {| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | ||
|scope="row" width="1000" style="text-align: left; background: #FFF8C6"| | |scope="row" width="1000" style="text-align: left; background: #FFF8C6"| | ||
− | '''1. Determine that the usage of the features in the [http://casting3.hopto.org application] are consistent to the expectations of the users (acting as of Casting Managers & Artistes)'''<br><br> | + | '''1. Determine that the usage of the features in the [http://dev-casting3.hopto.org/ application] are consistent to the expectations of the users (acting as of Casting Managers & Artistes)'''<br><br> |
'''2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction) and aesthetics of our application''' | '''2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction) and aesthetics of our application''' | ||
|} | |} | ||
Line 135: | Line 135: | ||
|} | |} | ||
− | <br> | + | <br><div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> |
===<font color="#4D268F">Session 1 (Casting Managers)</font>=== | ===<font color="#4D268F">Session 1 (Casting Managers)</font>=== | ||
Line 281: | Line 281: | ||
View our [https://docs.google.com/spreadsheet/ccc?key=0AskT4ENrBtHIdDNkOVdVWllrM0tyX3BqMHdCRkh1akE#gid=1 Response Plan to Feedback (Artistes)] | View our [https://docs.google.com/spreadsheet/ccc?key=0AskT4ENrBtHIdDNkOVdVWllrM0tyX3BqMHdCRkh1akE#gid=1 Response Plan to Feedback (Artistes)] | ||
|} | |} | ||
− | <br><br> | + | <br><br><div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> |
===<font color="#4D268F">Testing Methodology</font>=== | ===<font color="#4D268F">Testing Methodology</font>=== | ||
Line 303: | Line 303: | ||
|} | |} | ||
<br> | <br> | ||
+ | <div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> | ||
===<font color="#4D268F">Supporting Documents UT1</font>=== | ===<font color="#4D268F">Supporting Documents UT1</font>=== | ||
Line 354: | Line 355: | ||
{| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | {| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | ||
|scope="row" width="1000" style="text-align: left; background: #FFF8C6"| | |scope="row" width="1000" style="text-align: left; background: #FFF8C6"| | ||
− | '''1. Determine that the usage of the features in the [http://casting3.hopto.org application] are consistent to the expectations of <ins><font color="#D00000 ">real</font></ins> users (Casting Managers & Artistes)'''<br><br> | + | '''1. Determine that the usage of the features in the [http://dev-casting3.hopto.org/ application] are consistent to the expectations of <ins><font color="#D00000 ">real</font></ins> users (Casting Managers & Artistes)'''<br><br> |
'''2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction) and aesthetics of our application''' | '''2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction) and aesthetics of our application''' | ||
|}<br> | |}<br> | ||
===<font color="#4D268F">Scope of Test</font>=== | ===<font color="#4D268F">Scope of Test</font>=== | ||
+ | The scope of the test has been expanded from Usability Test 1 to include 6 more features: | ||
+ | |||
{| class="wikitable" style="text-align: center; height:50px" | {| class="wikitable" style="text-align: center; height:50px" | ||
|+ | |+ | ||
Line 405: | Line 408: | ||
! scope="row" style="background-color:#FBF8C7"| 6 | ! scope="row" style="background-color:#FBF8C7"| 6 | ||
− | |style="text-align: left;"|Accept/ reject an invitation | + | |style="text-align: left; background-color:#FBF8C7; "|Accept/ reject an invitation |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
|- | |- | ||
! scope="row" style="background-color:#FBF8C7"| 7 | ! scope="row" style="background-color:#FBF8C7"| 7 | ||
− | |style="text-align: left;"|Select an audition/ View audition date | + | |style="text-align: left; background-color:#FBF8C7;"|Select an audition/ View audition date |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|- | + | |style="text-align: center;background-color:#FBF8C7; "|- |
|- | |- | ||
Line 426: | Line 429: | ||
! scope="row" style="background-color:#FBF8C7"| 9 | ! scope="row" style="background-color:#FBF8C7"| 9 | ||
− | |style="text-align: left;"|Invite Casting Manager | + | |style="text-align: left; background-color:#FBF8C7"|Invite Casting Manager |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
|- | |- | ||
! scope="row" style="background-color:#FBF8C7"| 10 | ! scope="row" style="background-color:#FBF8C7"| 10 | ||
− | |style="text-align: left;"|View / Edit Casting Manager Portfolio | + | |style="text-align: left; background-color:#FBF8C7"|View / Edit Casting Manager Portfolio |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
|- | |- | ||
Line 461: | Line 464: | ||
! scope="row" style="background-color:#FBF8C7"| 14 | ! scope="row" style="background-color:#FBF8C7"| 14 | ||
− | |style="text-align: left;"|Create / Edit / View Auditions | + | |style="text-align: left; background-color:#FBF8C7"|Create / Edit / View Auditions |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
|- | |- | ||
! scope="row" style="background-color:#FBF8C7"| 15 | ! scope="row" style="background-color:#FBF8C7"| 15 | ||
− | |style="text-align: left;"|Manage Account | + | |style="text-align: left; background-color:#FBF8C7; "|Manage Account |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
− | |style="text-align: center;"|- | + | |style="text-align: center; background-color:#FBF8C7; "|- |
− | |style="text-align: center;"|[[Image:Greentick.png|24px]] | + | |style="text-align: center; background-color:#FBF8C7; "|[[Image:Greentick.png|24px]] |
|} | |} | ||
<br> | <br> | ||
+ | <div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> | ||
− | ===<font color="#4D268F"> | + | ===<font color="#4D268F">Plan for Usability Test 2</font>=== |
<!--[[Image:UT2_Structure.png|700px]]--> | <!--[[Image:UT2_Structure.png|700px]]--> | ||
<br> | <br> | ||
+ | |||
+ | initially, we planned to have both a lab test and field test for Usability Test 2. The lab test will be a more controlled one, whereby we could sit beside the users and observe their usage of our system. The field test, however, will be carried out at the actual environment (e.g. artistes at their homes and casting managers at their offices) to determine if they are capable of using the system at their own comfort without our guidance and interference. <br/> | ||
+ | |||
+ | Also, we wanted to carry out a pilot test before the actual test to verify if our planned test goes according to our expectations, and to make any necessary changes before the actual test. <br/> | ||
+ | |||
+ | The following chart shows our original plan for Usability Test 2: <br/><br/> | ||
[[Image:TW_UT2Actual.png|700px]] | [[Image:TW_UT2Actual.png|700px]] | ||
− | ===<font color="#4D268F"> | + | <div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> |
+ | |||
+ | ===<font color="#4D268F">Actual test and results</font>=== | ||
+ | However, as there wasn't any ongoing casting calls at the point where we carried out Usability Test 2, we decided to proceed with only a lab test and leave the field test for later. With Oak3's help and support, we gathered artistes with the following profiles: <br/> | ||
<br> | <br> | ||
[[Image:TW UT2 participants.png|300px]][[Image:TW_UT2_combined.jpg|700px|right]] | [[Image:TW UT2 participants.png|300px]][[Image:TW_UT2_combined.jpg|700px|right]] | ||
<br> | <br> | ||
+ | |||
+ | <div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> | ||
===<font color="#4D268F">Testing Methodology</font>=== | ===<font color="#4D268F">Testing Methodology</font>=== | ||
{| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | {| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | ||
|scope="row" width="900" style="text-align: left; background: #FFF8C6"| | |scope="row" width="900" style="text-align: left; background: #FFF8C6"| | ||
− | ''' | + | '''Collecting qualitative metrics (Lab Test)'''<br> |
:Users are encouraged to think out loud while accomplishing each task so as to better allow the facilitators to record down accurate observations and significant details of the testers’ thought processes. Facilitators will also be observing for errors (e.g. false clicks) and navigation issues. Upon completion of all the tasks, the tester will be presented a questionnaire in which he will be asked what he dislikes / likes about the application and their satisfaction level of the application. | :Users are encouraged to think out loud while accomplishing each task so as to better allow the facilitators to record down accurate observations and significant details of the testers’ thought processes. Facilitators will also be observing for errors (e.g. false clicks) and navigation issues. Upon completion of all the tasks, the tester will be presented a questionnaire in which he will be asked what he dislikes / likes about the application and their satisfaction level of the application. | ||
[[Image:1351780541_27-Edit Text.png|right]]<br> | [[Image:1351780541_27-Edit Text.png|right]]<br> | ||
Line 496: | Line 511: | ||
:The testing process will also be recorded via video using '''[http://www.telestream.net/screenflow/overview.htm Screenflow]''' and will complement the facilitators’ observations. Facilitators are to record the timing in which a note-worthy observation (e.g. body language response) occurred. | :The testing process will also be recorded via video using '''[http://www.telestream.net/screenflow/overview.htm Screenflow]''' and will complement the facilitators’ observations. Facilitators are to record the timing in which a note-worthy observation (e.g. body language response) occurred. | ||
− | |||
− | |||
− | |||
|}<br> | |}<br> | ||
Line 513: | Line 525: | ||
|style="text-align: left;"| | |style="text-align: left;"| | ||
:1. '''[https://docs.google.com/document/d/1SMrs7RGpbMa0RRNFAC8Nqsa0OfPPfd28ejGwAu83ckM/edit UT2 Plan]''' | :1. '''[https://docs.google.com/document/d/1SMrs7RGpbMa0RRNFAC8Nqsa0OfPPfd28ejGwAu83ckM/edit UT2 Plan]''' | ||
− | :2. '''[https://docs.google.com/spreadsheet/ccc?key=0AgWbd4V2n6c-dHY4clprVjZ4UTNucVZsVW9rN1UwWXc#gid=0 Artiste Observed Feedback]''' | + | :2. '''[https://docs.google.com/spreadsheet/ccc?key=0AgWbd4V2n6c-dHY4clprVjZ4UTNucVZsVW9rN1UwWXc#gid=0 Artiste Observed Feedback and Response Plan]''' |
<!--:3. '''[https://docs.google.com/document/d/1mJJ0f2Jb_arU57sMEQZKS-TtE8pNITMYi7eNEVYM8sc/edit Artiste Survey Template]'''--> | <!--:3. '''[https://docs.google.com/document/d/1mJJ0f2Jb_arU57sMEQZKS-TtE8pNITMYi7eNEVYM8sc/edit Artiste Survey Template]'''--> | ||
− | :3. '''[http://edu.surveygizmo.com/s3/1065819/Artiste-Feedback Artiste Survey | + | :3. '''[http://edu.surveygizmo.com/s3/1065819/Artiste-Feedback Artiste Survey Link]''' |
:4. '''[https://docs.google.com/file/d/0BzUq1d4sV8JQRkxVRVBpeDV1SGM/edit Artiste Survey Results Report]''' | :4. '''[https://docs.google.com/file/d/0BzUq1d4sV8JQRkxVRVBpeDV1SGM/edit Artiste Survey Results Report]''' | ||
+ | :5. '''[https://docs.google.com/spreadsheet/ccc?key=0AhS1AYvg7B7DdGF1OGxGelQ2bzljOWhTZUZ5U0p6X1E#gid=1 Casting Manager Observed Feedback and Response Plan]''' | ||
+ | :6. '''[https://docs.google.com/document/d/1pvLflUVvNzhR8btjHcYnIhJ_2d0rQ8Dt4OSX10usboE/edit Casting Manager Task List]''' | ||
+ | :7. '''[https://docs.google.com/document/d/1PDfg0I8LtbgCiJJCG7jk5KG63q0_iyc6eTOdnjQ0N5g/edit Artiste Task List]''' | ||
+ | |} | ||
+ | <br> | ||
+ | |||
+ | ===<font color="#4D268F">Extract of Solutions to Most Common Feedback</font>=== | ||
+ | [[Image:UT21.png|900px]]<br><br><br> | ||
+ | [[Image:TW_UT22.png|900px]] | ||
+ | |||
+ | <div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_UsabilityTest#Top|Back to Top]]</div> | ||
+ | |}<br/><br/> | ||
+ | |||
+ | ==<font color="#c93756">'''Field Test'''</font>== | ||
+ | {| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | ||
+ | |scope="row" width="1200" style="text-align: left; background: #F5F5F5"| | ||
+ | |||
+ | ====<font color="#c93756">Objectives</font>==== | ||
+ | {| cellpadding="9" style="border: 1px solid darkgray; text-align: center; height:50px" | ||
+ | |scope="row" width="1000" style="text-align: left; background: #FFF8C6"| | ||
+ | '''1. Determine if users are capable of handling the application on their own (without any guidance from us) in the <ins><font color="#D00000 ">actual environment</font></ins> where they would be using the application '''<br><br> | ||
+ | '''2. Obtain feedback from users such as to improve the usability (learnability, efficiency, errors, satisfaction) and aesthetics of our application''' | ||
+ | |}<br> | ||
+ | <br> | ||
+ | ====<font color="#c93756">Field Test Details & Outcome</font>==== | ||
− | | | + | <br> |
+ | Field Test was conducted from '''28 November 2012 to 1 December 2012''' remotely at the artistes' own comfort. With the help and support of Oak3, we managed to get some artistes to sign up as a user of Casting3, and to apply for an ongoing casting call, "Koffee did it". They will be provided with a link to an evaluation form, where they will asked to give us their feedback. | ||
+ | <br/><br/> | ||
+ | [[Image:TW FieldTestMethod.png|1024px]]<br/> | ||
+ | |||
+ | As of the submission date of this wiki page (28 Nov), the field test is ongoing and results have yet to be collated. The results will be later concluded in the following links:<br><br> | ||
+ | |||
+ | '''Feedback Links:''' | ||
+ | :'''1. [https://docs.google.com/document/d/1c3bWbgZeYfdbZ5eyUqxEhOBi0XH7mfWhLNkgPBbECx0/edit Profile of artistes who have signed up]'''<br> | ||
+ | :'''2. [https://docs.google.com/document/d/152I81O5K94e8Unraf0fHUTPWFDbZDWiz82mtqCvY1Ns/edit Survey for Artistes and their feedback]'''<br> | ||
+ | :'''3. [https://docs.google.com/spreadsheet/ccc?key=0AskT4ENrBtHIdFpBTWNCVjF1bDZNZVI2NmNjMnpKVUE#gid=0 Response Plan to Artiste Feedback]'''<br> | ||
+ | |||
+ | |||
+ | <div style = "text-align:right; padding: 0px 30px 0px 45px;">[[IS480_Team_wiki:_2012T1_Timber_Werkz_Final_Wiki#Top|Back to Top]]</div> | ||
+ | |||
+ | <br><br> |
Latest revision as of 23:50, 28 November 2012
HOME | PROJECT OVERVIEW | PROJECT MANAGEMENT | DOCUMENTATION |
<< MID-TERM WIKI << FINAL WIKI
Usability Test 1
Objectives
Scope of TestSession 1 (Casting Managers)
Session 2 (Artistes)Session 2 was conducted from 9 - 14 Sept 2012
Testing Methodology
Supporting Documents UT1
|
Usability Test 2
Objectives
Scope of TestThe scope of the test has been expanded from Usability Test 1 to include 6 more features:
Plan for Usability Test 2
initially, we planned to have both a lab test and field test for Usability Test 2. The lab test will be a more controlled one, whereby we could sit beside the users and observe their usage of our system. The field test, however, will be carried out at the actual environment (e.g. artistes at their homes and casting managers at their offices) to determine if they are capable of using the system at their own comfort without our guidance and interference. Also, we wanted to carry out a pilot test before the actual test to verify if our planned test goes according to our expectations, and to make any necessary changes before the actual test. The following chart shows our original plan for Usability Test 2: Actual test and resultsHowever, as there wasn't any ongoing casting calls at the point where we carried out Usability Test 2, we decided to proceed with only a lab test and leave the field test for later. With Oak3's help and support, we gathered artistes with the following profiles:
Testing Methodology
Supporting Documents UT2
Extract of Solutions to Most Common Feedback |
Field Test
Objectives
Field Test Details & Outcome
As of the submission date of this wiki page (28 Nov), the field test is ongoing and results have yet to be collated. The results will be later concluded in the following links: Feedback Links:
|