Difference between revisions of "IS480 Team wiki: 2013T2 LittleTeam ProjectDocumentation Testing"
Jump to navigation
Jump to search
Qian.ye.2011 (talk | contribs) |
Qian.ye.2011 (talk | contribs) |
||
(34 intermediate revisions by 3 users not shown) | |||
Line 30: | Line 30: | ||
=='''User Testing 1'''== | =='''User Testing 1'''== | ||
− | < | + | <div align=left style="padding-right:50px; padding-left:50px;"> |
− | '''Location''': LittleLives | + | <font size=3 color="000099">'''Date''': 20 Dec 2013 |
+ | <br/>'''Location''': LittleLives | ||
<br/>'''Participant Group''': LittleLives Staff | <br/>'''Participant Group''': LittleLives Staff | ||
<br/>'''Participant Size''': 8 | <br/>'''Participant Size''': 8 | ||
</font> | </font> | ||
− | < | + | </div> |
+ | |||
===Testing Documentations=== | ===Testing Documentations=== | ||
{| class="wikitable" style="text-align: center; height:50px" | {| class="wikitable" style="text-align: center; height:50px" | ||
Line 117: | Line 119: | ||
=='''User Testing 2'''== | =='''User Testing 2'''== | ||
− | < | + | <div align=left style="padding-right:50px; padding-left:50px;"> |
+ | <font size=3 color="000099">'''Date''': 18 Jan 2014 | ||
'''Location''': The Learning Lab (United Square) | '''Location''': The Learning Lab (United Square) | ||
<br/>'''Participant Group''': Secondary Grade 2 - 4 Students | <br/>'''Participant Group''': Secondary Grade 2 - 4 Students | ||
<br/>'''Participant Size''': 12; | <br/>'''Participant Size''': 12; | ||
</font> | </font> | ||
− | < | + | </div> |
===Testing Documentations=== | ===Testing Documentations=== | ||
{| class="wikitable" style="text-align: center; height:50px" | {| class="wikitable" style="text-align: center; height:50px" | ||
Line 159: | Line 162: | ||
Image:IS480 LittleTeam UT2 AvgnCLC.png | Image:IS480 LittleTeam UT2 AvgnCLC.png | ||
</gallery> | </gallery> | ||
+ | ====Qualitative Results==== | ||
+ | {| class="wikitable" style="text-align: center; height:50px" | ||
+ | |+ | ||
+ | |- | ||
+ | ! scope="col" width="550" style="background:#006633; color:#ffffff"|Top Comments | ||
+ | ! scope="col" width="550" style="background:#006633; color:#ffffff"|Potential Problems & Improvements | ||
+ | |- | ||
+ | |||
+ | |scope="row" style=" text-align: left;"| "The 3 horizontal lines don’t really clearly inform me of its purpose" | ||
+ | |scope="row" style=" text-align: left;"| Remove the horizontal lines and standardize the menu format | ||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"| "There should be some instructions on choosing the questions for the knowledge challenge" | ||
+ | |scope="row" style=" text-align: left;"| Add in step-by-step guides for users when sending the challenge | ||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"| "The landing page of Knowledge challenge is too blank and confusing for the first time user" | ||
+ | |scope="row" style=" text-align: left;"| Change the layout of the landing page of knowledge challenge. | ||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"| "Very hard to find where to search friend" | ||
+ | |scope="row" style=" text-align: left;"| Standardize the top tool bar and merge the search function in. | ||
+ | |||
+ | |- | ||
+ | ! scope="col" width="550" style="background:blue; color:white"|Generally the system is hard to use for the first time user. Quite a few unnecessary pages and icons. | ||
+ | ! scope="col" width="550" style="background:blue; color:white"|UI Revamp seems necessary, but need to further discuss with client and supervisor | ||
+ | |||
+ | |} | ||
+ | |||
====Comparing with UT1==== | ====Comparing with UT1==== | ||
<gallery widths=450px heights=350px perrow=3 mode ="packed"> | <gallery widths=450px heights=350px perrow=3 mode ="packed"> | ||
Line 167: | Line 196: | ||
=='''User Testing 3'''== | =='''User Testing 3'''== | ||
− | + | <div align=left style="padding-right:50px; padding-left:50px;"> | |
+ | <font size=3 color="000099">'''Date''': 15 Feb 2014 | ||
+ | '''Location''': The Learning Lab (United Square) | ||
+ | <br/>'''Participant Group''': Primary 5 - 6 Students | ||
+ | <br/>'''Participant Size''': 12; | ||
+ | <br/>'''Testing Method''': A/B Testing. This to... | ||
+ | *See the user preference on the side/top tool bar | ||
+ | *Test the difference between these two options statistically. | ||
+ | <br/>2 systems are identical except for... | ||
+ | * '''System A''' has only the side tool bar. Refer to [http://sysa.littleteamsmu.com/ here]. | ||
+ | * '''System B''' has only the top tool bar. Refer to [http://sysb.littleteamsmu.com/ here]. | ||
+ | </font> | ||
+ | </div> | ||
===Testing Documentations=== | ===Testing Documentations=== | ||
{| class="wikitable" style="text-align: center; height:50px" | {| class="wikitable" style="text-align: center; height:50px" | ||
Line 202: | Line 243: | ||
Image:IS480 LittleTeam UT3 AvgnnCLD.png | Image:IS480 LittleTeam UT3 AvgnnCLD.png | ||
</gallery> | </gallery> | ||
+ | |||
====T-test Analysis==== | ====T-test Analysis==== | ||
<gallery widths=450px heights=350px perrow=3 mode ="packed"> | <gallery widths=450px heights=350px perrow=3 mode ="packed"> | ||
Image:IS480 LittleTeam UT3 t-test.PNG | Image:IS480 LittleTeam UT3 t-test.PNG | ||
</gallery> | </gallery> | ||
− | < | + | <div align=left style="padding-right:50px; padding-left:50px;"> |
<font size=3.2> | <font size=3.2> | ||
Since two different groups of participants test our System A and System B, therefore we ran the statistical standard T-Test for between subjects. | Since two different groups of participants test our System A and System B, therefore we ran the statistical standard T-Test for between subjects. | ||
<br/> | <br/> | ||
− | *Test: Two-tailed t-test at 95% significance level (α=5%) | + | *Test: Two-tailed t-test at 95% significance level (α=5%), 2 Sample assuming equal variance |
*Independent Variable: System A vs. System B | *Independent Variable: System A vs. System B | ||
*Dependent Variable: Mean time taken to complete the Ask Question task | *Dependent Variable: Mean time taken to complete the Ask Question task | ||
Line 216: | Line 258: | ||
**Let μ1 be the mean time to complete the task using System B(treatment). | **Let μ1 be the mean time to complete the task using System B(treatment). | ||
<br/> | <br/> | ||
− | *Hypotheses: | + | *Hypotheses: |
− | **Null Hypothesis h0: | + | **Null Hypothesis h0: μ0=μ1 |
− | **Alternative Hypothesis: h1: | + | **Alternative Hypothesis: h1: μ0≠μ1 |
<br/> | <br/> | ||
*P-value: | *P-value: | ||
− | **As calculated using Excel formula: T.Test(array1,array2,2,3), p≈0.02= | + | **As calculated using Excel formula: T.Test(array1,array2,2,3), p≈0.02=2% |
**'''p < α''', then significant, not by chance. | **'''p < α''', then significant, not by chance. | ||
− | **Therefore we do | + | **Therefore we do reject H0. |
<br/> | <br/> | ||
− | *Conclusion: H0 is true and thus the mean time taken in completing the tasks is significantly different, not by chance. | + | *Conclusion: H0 is not true and thus the mean time taken in completing the tasks is significantly different, not by chance. |
</font> | </font> | ||
− | <center | + | </div> |
+ | |||
+ | ====Qualitative Results==== | ||
+ | {| class="wikitable" style="text-align: center; height:50px" | ||
+ | |+ | ||
+ | |- | ||
+ | ! scope="col" width="550" style="background:#006633; color:#ffffff"|Top Problems Observed | ||
+ | ! scope="col" width="550" style="background:#006633; color:#ffffff"|Improvements | ||
+ | |- | ||
+ | |||
+ | |scope="row" style=" text-align: left;"| Calculation of the percentage of voting shows 101%, 1% which are not logically correct | ||
+ | |scope="row" style=" text-align: left;"| Change the calculation methods in back-end. | ||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"| When sending collaborative challenge, students tend to select 3 (same as sending individual challenge) instead of 1. | ||
+ | |scope="row" style=" text-align: left;"| The reminding note is not obvious, make the notes more obvious to users | ||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"| When sending challenge to individual person, user tend to fill in all the fields which are prepared for sending challenge to group only. | ||
+ | |scope="row" style=" text-align: left;"| change the forms format to different tabs format to differentiate"To Another User"/"To another Class"/"To another School" | ||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"|The categories pictures are not clickable. | ||
+ | |scope="row" style=" text-align: left;"| Make the pictures been able to click. | ||
+ | |||
+ | |- | ||
+ | ! scope="col" width="550" style="background:blue; color:#ffffff"|Left-side bar is seldom used by the users. They tend to go back to home page and click on the modules or top bar | ||
+ | ! scope="col" width="550" style="background:blue; color:#ffffff"|Together with the quantitative results and statistical analysis, it is highly suggested to keep the top tool bar instead of the side tool bar | ||
+ | |||
+ | |||
+ | |} | ||
====Comparing with UT2==== | ====Comparing with UT2==== | ||
Line 234: | Line 303: | ||
Image:IS480 LittleTeam UT3vsUT2 22.png | Image:IS480 LittleTeam UT3vsUT2 22.png | ||
Image:IS480 LittleTeam UT3vsUT2 3.png | Image:IS480 LittleTeam UT3vsUT2 3.png | ||
+ | </gallery> | ||
+ | |||
+ | =='''User Testing 4'''== | ||
+ | <div align=left style="padding-right:50px; padding-left:50px;"> | ||
+ | <font size=3 color="000099">'''Date''': 4 Apr 2014 <br/> | ||
+ | '''Location''': Sengkang Public Library | ||
+ | <br/>'''Participant Group''': Primary School P5 students | ||
+ | <br/>'''Participant Size''': 13; | ||
+ | </font> | ||
+ | </div> | ||
+ | ===Testing Documentations=== | ||
+ | {| class="wikitable" style="text-align: center; height:50px" | ||
+ | |+ | ||
+ | |- | ||
+ | ! scope="col" width="250" style="background:#FFCC33; color:#333333"| Goals&Tasks&Data | ||
+ | ! scope="col" width="250" style="background:#FFCC33; color:#333333"| Scripts | ||
+ | ! scope="col" width="250" style="background:#FFCC33; color:#333333"| Instructions - Set 1 | ||
+ | ! scope="col" width="250" style="background:#FFCC33; color:#333333"| Instructions - Set 2 | ||
+ | ! scope="col" width="250" style="background:#FFCC33; color:#333333"| Observation Forms | ||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |[[Media:LittleTeam - UT 4 - goals & data & tasks.docx |Download]] | ||
+ | |[[Media:LittleTeam - UT 4 - Testing Script.docx |Download]] | ||
+ | |[[Media:LittleTeam - UT 4 - Instructions&feedback - Set1.docx |Download]] | ||
+ | |[[Media:LittleTeam - UT 4 - Instructions&feedback - Set1.docx |Download]] | ||
+ | |[[Media:LittleTeam - UT 4 - observation form.xlsx|Download]] | ||
+ | |- | ||
+ | |} | ||
+ | |||
+ | ===Testing in Actions=== | ||
+ | <gallery widths=350px heights=250px perrow=3 mode ="packed"> | ||
+ | LittlteTeam UT4 - photo1.JPG | ||
+ | LittleTeam UT4 -photo2.JPG | ||
+ | |||
+ | </gallery> | ||
+ | |||
+ | ===Testing Results=== | ||
+ | ====Quantitative Results==== | ||
+ | <gallery widths=450px heights=350px perrow=3 mode ="packed"> | ||
+ | LittleTeam- UT4- charts2.png | ||
+ | LittleTeam- UT4- charts3.png | ||
+ | LittleTeam- UT4- charts4.png | ||
+ | LittleTeam- UT4- charts5.png | ||
+ | LittleTeam- UT4- charts6.png | ||
+ | LittleTeam- UT4- charts7.png | ||
+ | </gallery> | ||
+ | |||
+ | ====Qualitative Results==== | ||
+ | {| class="wikitable" style="text-align: center; height:50px" | ||
+ | |+ | ||
+ | |- | ||
+ | ! scope="col" width="550" style="background:#006633; color:#ffffff"|Top Problems Observed | ||
+ | ! scope="col" width="550" style="background:#006633; color:#ffffff"|Improvements | ||
+ | |- | ||
+ | |||
+ | |scope="row" style=" text-align: left;"| When creating a lesson in Little Classroom, the different colours of buttons are distracting for the users. It took them some time to figure out the functionalities of different buttons | ||
+ | |scope="row" style=" text-align: left;"| Simplify and standardize the buttons in Little Classroom. | ||
+ | |||
+ | |- | ||
+ | |scope="row" style=" text-align: left;"| When the user is in the specific lesson page, the tab of the browser shows "Knowledge Challenge" instead. | ||
+ | |scope="row" style=" text-align: left;"| Resolve the minor aesthetic bug. | ||
+ | |||
+ | |} | ||
+ | |||
+ | ====Comparing with UT3==== | ||
+ | <gallery widths=450px heights=350px perrow=3 mode ="packed"> | ||
+ | LittleTeam- UT4- charts10.png | ||
+ | LittleTeam- UT4- charts11.png | ||
+ | LittleTeam- UT4- charts12.png | ||
+ | </gallery> | ||
+ | |||
+ | ====Comparing with Previous UTs==== | ||
+ | <gallery widths=450px heights=350px perrow=3 mode ="packed"> | ||
+ | LittleTeam- UT4- charts8.png | ||
+ | LittleTeam- UT4- charts9.png | ||
+ | LittleTeam- UT4- charts1.png | ||
</gallery> | </gallery> | ||
Line 285: | Line 431: | ||
|- | |- | ||
+ | |- | ||
+ | !scope="row" width="300" style=" text-align: center;"| Sprint 9 | ||
+ | |[[Media:IS480 LittleTeam TestCases Sprint9.xlsx|Download]] | ||
+ | |- | ||
+ | |||
+ | |- | ||
+ | !scope="row" width="300" style=" text-align: center;"| Sprint 10 | ||
+ | |[[Media:IS480 LittleTeam TestCases Sprint10.xlsx|Download]] | ||
+ | |- | ||
+ | |||
+ | |- | ||
+ | !scope="row" width="300" style=" text-align: center;"| Sprint 11 | ||
+ | |[[Media:LittleTeam TestCases Sprint11.xlsx|Download]] | ||
+ | |- | ||
+ | |||
+ | |- | ||
+ | !scope="row" width="300" style=" text-align: center;"| Sprint 12 | ||
+ | |[[Media:LittleTeam TestCases Sprint12.xlsx|Download]] | ||
+ | |- | ||
|} | |} |
Latest revision as of 16:24, 14 April 2014
Home | The Team | Project Overview | Project Management | Project Documentation | Team Resources | Team Reflection |
---|
UI/UX Design | System Design | Testing | Administrative |
---|
Contents
Persona
Primary Persona
Secondary Persona
User Testing 1
Date: 20 Dec 2013
Location: LittleLives
Participant Group: LittleLives Staff
Participant Size: 8
Testing Documentations
Goals&Tasks&Data | Scripts | Instructions | Observation Forms | Questionnaires |
---|---|---|---|---|
Download | Download | Download | Download | Link |
Testing In Actions
Testing Results
Quantitative Data - Time taken to finish the task |
---|
Quantitative Data - No. of clicks to finish the task |
---|
Qualitative Data - Usability Rating |
---|
Frequently Seen Comments | Potential Problems & Improvements |
---|---|
"I could not find where to view my score" | The score display is not available and the score board design is not properly aligned. |
"The success messages are not obvious enough for the users" | Change the font size or color of success messages. |
"Accept and Reject buttons are not readable" | Change the font size and color of the buttons |
"The rules for knowledge are not easy and hard to remember" | Will implement 'Help' in later sprint when major functions are done |
"The navigation bar in Knowledge Challenge takes up too many spaces and is not professional" | Change to hidden navigation bar at bottom |
"There are too many unnecessary information in Scoreboard, which makes the overall design very crowded" | Limited the information displayed in Scoreboard and shift it to the left of the screen |
"I cannot view my status of challenges that I answered previously" | Create a new page, which displays "Challenge Status" |
User Testing 2
Date: 18 Jan 2014
Location: The Learning Lab (United Square)
Participant Group: Secondary Grade 2 - 4 Students
Participant Size: 12;
Testing Documentations
Goals&Tasks&Data | Scripts | Instructions - Set 1 | Instructions - Set 2 | Observation Forms |
---|---|---|---|---|
Download | Download | Download | Download | Download |
Testing In Actions
Testing Results
Quantitative Results
Qualitative Results
Top Comments | Potential Problems & Improvements |
---|---|
"The 3 horizontal lines don’t really clearly inform me of its purpose" | Remove the horizontal lines and standardize the menu format |
"There should be some instructions on choosing the questions for the knowledge challenge" | Add in step-by-step guides for users when sending the challenge |
"The landing page of Knowledge challenge is too blank and confusing for the first time user" | Change the layout of the landing page of knowledge challenge. |
"Very hard to find where to search friend" | Standardize the top tool bar and merge the search function in. |
Generally the system is hard to use for the first time user. Quite a few unnecessary pages and icons. | UI Revamp seems necessary, but need to further discuss with client and supervisor |
Comparing with UT1
User Testing 3
Date: 15 Feb 2014
Location: The Learning Lab (United Square)
Participant Group: Primary 5 - 6 Students
Participant Size: 12;
Testing Method: A/B Testing. This to...
- See the user preference on the side/top tool bar
- Test the difference between these two options statistically.
2 systems are identical except for...
- System A has only the side tool bar. Refer to here.
- System B has only the top tool bar. Refer to here.
Testing Documentations
Goals&Tasks&Data | Scripts | Instructions | Observation Forms |
---|---|---|---|
Download | Download | Download | Download |
Testing in Actions
Testing Results
Quantitative Results
T-test Analysis
Since two different groups of participants test our System A and System B, therefore we ran the statistical standard T-Test for between subjects.
- Test: Two-tailed t-test at 95% significance level (α=5%), 2 Sample assuming equal variance
- Independent Variable: System A vs. System B
- Dependent Variable: Mean time taken to complete the Ask Question task
- Let μ0 be the mean time to complete the task using System A(control).
- Let μ1 be the mean time to complete the task using System B(treatment).
- Hypotheses:
- Null Hypothesis h0: μ0=μ1
- Alternative Hypothesis: h1: μ0≠μ1
- P-value:
- As calculated using Excel formula: T.Test(array1,array2,2,3), p≈0.02=2%
- p < α, then significant, not by chance.
- Therefore we do reject H0.
- Conclusion: H0 is not true and thus the mean time taken in completing the tasks is significantly different, not by chance.
Qualitative Results
Top Problems Observed | Improvements |
---|---|
Calculation of the percentage of voting shows 101%, 1% which are not logically correct | Change the calculation methods in back-end. |
When sending collaborative challenge, students tend to select 3 (same as sending individual challenge) instead of 1. | The reminding note is not obvious, make the notes more obvious to users |
When sending challenge to individual person, user tend to fill in all the fields which are prepared for sending challenge to group only. | change the forms format to different tabs format to differentiate"To Another User"/"To another Class"/"To another School" |
The categories pictures are not clickable. | Make the pictures been able to click. |
Left-side bar is seldom used by the users. They tend to go back to home page and click on the modules or top bar | Together with the quantitative results and statistical analysis, it is highly suggested to keep the top tool bar instead of the side tool bar
|
Comparing with UT2
User Testing 4
Date: 4 Apr 2014
Location: Sengkang Public Library
Participant Group: Primary School P5 students
Participant Size: 13;
Testing Documentations
Goals&Tasks&Data | Scripts | Instructions - Set 1 | Instructions - Set 2 | Observation Forms |
---|---|---|---|---|
Download | Download | Download | Download | Download |
Testing in Actions
Testing Results
Quantitative Results
Qualitative Results
Top Problems Observed | Improvements |
---|---|
When creating a lesson in Little Classroom, the different colours of buttons are distracting for the users. It took them some time to figure out the functionalities of different buttons | Simplify and standardize the buttons in Little Classroom. |
When the user is in the specific lesson page, the tab of the browser shows "Knowledge Challenge" instead. | Resolve the minor aesthetic bug. |
Comparing with UT3
Comparing with Previous UTs
Internal Testing
Sprint | Test Cases |
---|---|
Sprint 1 | Download |
Sprint 2 | Download |
Sprint 3 | Download |
Sprint 4 | Download |
Sprint 5 | Download |
Sprint 6 | Download |
Sprint 7 | Download |
Sprint 8 | Download |
Sprint 9 | Download |
Sprint 10 | Download |
Sprint 11 | Download |
Sprint 12 | Download |