HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2013T2 LittleTeam ProjectDocumentation Testing"

From IS480
Jump to navigation Jump to search
 
(48 intermediate revisions by 3 users not shown)
Line 30: Line 30:
  
 
=='''User Testing 1'''==
 
=='''User Testing 1'''==
</center> <font size=3 color="000099">'''Date''': 20 Dec 2013
+
<div align=left style="padding-right:50px; padding-left:50px;">
'''Location''': LittleLives  
+
<font size=3 color="000099">'''Date''': 20 Dec 2013
 +
<br/>'''Location''': LittleLives  
 
<br/>'''Participant Group''': LittleLives Staff
 
<br/>'''Participant Group''': LittleLives Staff
 
<br/>'''Participant Size''': 8   
 
<br/>'''Participant Size''': 8   
 
</font>
 
</font>
<center>
+
</div>
 +
 
 
===Testing Documentations===
 
===Testing Documentations===
 
{| class="wikitable" style="text-align: center; height:50px"
 
{| class="wikitable" style="text-align: center; height:50px"
Line 117: Line 119:
  
 
=='''User Testing 2'''==
 
=='''User Testing 2'''==
</center> <font size=3 color="000099">'''Date''': 18 Jan 2014
+
<div align=left style="padding-right:50px; padding-left:50px;">
 +
<font size=3 color="000099">'''Date''': 18 Jan 2014
 
'''Location''': The Learning Lab (United Square)  
 
'''Location''': The Learning Lab (United Square)  
 
<br/>'''Participant Group''': Secondary Grade 2 - 4 Students   
 
<br/>'''Participant Group''': Secondary Grade 2 - 4 Students   
 
<br/>'''Participant Size''': 12;
 
<br/>'''Participant Size''': 12;
 
</font>
 
</font>
<center>
+
</div>
 
===Testing Documentations===
 
===Testing Documentations===
 
{| class="wikitable" style="text-align: center; height:50px"
 
{| class="wikitable" style="text-align: center; height:50px"
Line 159: Line 162:
 
  Image:IS480 LittleTeam UT2 AvgnCLC.png  
 
  Image:IS480 LittleTeam UT2 AvgnCLC.png  
 
</gallery>
 
</gallery>
 +
====Qualitative Results====
 +
{| class="wikitable" style="text-align: center; height:50px"
 +
|+
 +
|-
 +
! scope="col"  width="550" style="background:#006633; color:#ffffff"|Top Comments
 +
! scope="col" width="550" style="background:#006633; color:#ffffff"|Potential Problems & Improvements 
 +
|-
 +
 +
|scope="row"  style=" text-align: left;"| "The 3 horizontal lines don’t really clearly inform me of its purpose"
 +
|scope="row"  style=" text-align: left;"| Remove the horizontal lines and standardize the menu format
 +
|-
 +
|scope="row"  style=" text-align: left;"| "There should be some instructions on choosing the questions for the knowledge challenge"
 +
|scope="row"  style=" text-align: left;"| Add in step-by-step guides for users when sending the challenge
 +
|-
 +
|scope="row"  style=" text-align: left;"| "The landing page of Knowledge challenge is too blank and confusing for the first time user"
 +
|scope="row"  style=" text-align: left;"| Change the layout of the landing page of knowledge challenge.
 +
|-
 +
|scope="row"  style=" text-align: left;"| "Very hard to find where to search friend"
 +
|scope="row"  style=" text-align: left;"| Standardize the top tool bar and merge the search function in.
 +
 +
|-
 +
! scope="col"  width="550" style="background:blue; color:white"|Generally the system is hard to use for the first time user. Quite a few unnecessary pages and icons.
 +
! scope="col" width="550" style="background:blue; color:white"|UI Revamp seems necessary, but need to further discuss with client and supervisor
 +
 +
|}
 +
 
====Comparing with UT1====
 
====Comparing with UT1====
 
<gallery widths=450px heights=350px perrow=3 mode ="packed">
 
<gallery widths=450px heights=350px perrow=3 mode ="packed">
Line 167: Line 196:
  
 
=='''User Testing 3'''==
 
=='''User Testing 3'''==
 
+
<div align=left style="padding-right:50px; padding-left:50px;">
 +
<font size=3 color="000099">'''Date''': 15 Feb 2014
 +
'''Location''': The Learning Lab (United Square)
 +
<br/>'''Participant Group''': Primary 5 - 6 Students 
 +
<br/>'''Participant Size''': 12;
 +
<br/>'''Testing Method''': A/B Testing. This to...
 +
*See the user preference on the side/top tool bar
 +
*Test the difference between these two options statistically.
 +
<br/>2 systems are identical except for...
 +
* '''System A''' has only the side tool bar. Refer to [http://sysa.littleteamsmu.com/ here].
 +
* '''System B''' has only the top tool bar. Refer to [http://sysb.littleteamsmu.com/  here].
 +
</font>
 +
</div>
 
===Testing Documentations===
 
===Testing Documentations===
 
{| class="wikitable" style="text-align: center; height:50px"
 
{| class="wikitable" style="text-align: center; height:50px"
Line 182: Line 223:
 
|[[Media:LittleTeam - UT 3 - Testing Script.docx|Download]]‎
 
|[[Media:LittleTeam - UT 3 - Testing Script.docx|Download]]‎
 
|[[Media:LittleTeam - UT 3 - Instructions - FULL.docx |Download]]
 
|[[Media:LittleTeam - UT 3 - Instructions - FULL.docx |Download]]
|[[Media:LittleTeam - UT 1 - observation form & Records.xlsx|Download]]
+
|[[Media:LittleTeam - UT 3 - observation form.xlsx‎|Download]]
 
|-
 
|-
 
|}
 
|}
  
===Testing in Actions===  
+
===Testing in Actions===
 +
<gallery widths=350px heights=250px perrow=3 mode ="packed">
 +
Image:IS480 LittleTeam UT3 photo1.JPG
 +
Image:IS480 LittleTeam UT3 photo3.jpg
 +
</gallery>
  
 
===Testing Results===
 
===Testing Results===
Line 198: Line 243:
 
  Image:IS480 LittleTeam UT3 AvgnnCLD.png  
 
  Image:IS480 LittleTeam UT3 AvgnnCLD.png  
 
</gallery>
 
</gallery>
 +
 +
====T-test Analysis====
 +
<gallery widths=450px heights=350px perrow=3 mode ="packed">
 +
Image:IS480 LittleTeam UT3 t-test.PNG
 +
</gallery>
 +
<div align=left style="padding-right:50px; padding-left:50px;">
 +
<font size=3.2>
 +
Since two different groups of participants test our System A and System B, therefore we ran the statistical standard T-Test for between subjects.
 +
<br/>
 +
*Test: Two-tailed t-test at 95% significance level (α=5%), 2 Sample assuming equal variance
 +
*Independent Variable: System A vs. System B
 +
*Dependent Variable: Mean time taken to complete the Ask Question task
 +
**Let μ0 be the mean time to complete the task using System A(control).
 +
**Let μ1 be the mean time to complete the task using System B(treatment).
 +
<br/>
 +
*Hypotheses:
 +
**Null Hypothesis h0: μ0=μ1
 +
**Alternative Hypothesis: h1:  μ0≠μ1
 +
<br/>
 +
*P-value:
 +
**As calculated using Excel formula: T.Test(array1,array2,2,3), p≈0.02=2%
 +
**'''p < α''', then significant, not by chance.
 +
**Therefore we do reject H0.
 +
<br/>
 +
*Conclusion: H0 is not true and thus the mean time taken in completing the tasks is significantly different, not by chance.
 +
</font>
 +
</div>
 +
 +
====Qualitative Results====
 +
{| class="wikitable" style="text-align: center; height:50px"
 +
|+
 +
|-
 +
! scope="col"  width="550" style="background:#006633; color:#ffffff"|Top Problems Observed
 +
! scope="col" width="550" style="background:#006633; color:#ffffff"|Improvements 
 +
|-
 +
 +
|scope="row"  style=" text-align: left;"| Calculation of the percentage of voting shows 101%, 1% which are not logically correct
 +
|scope="row"  style=" text-align: left;"| Change the calculation methods in back-end.
 +
|-
 +
|scope="row"  style=" text-align: left;"| When sending collaborative challenge, students tend to select 3 (same as sending individual challenge) instead of 1.
 +
|scope="row"  style=" text-align: left;"| The reminding note is not obvious, make the notes more obvious to users
 +
|-
 +
|scope="row"  style=" text-align: left;"| When sending challenge to individual person, user tend to fill in all the fields which are prepared for sending challenge to group only.
 +
|scope="row"  style=" text-align: left;"| change the forms format to different tabs format to differentiate"To Another User"/"To another Class"/"To another School"
 +
|-
 +
|scope="row"  style=" text-align: left;"|The categories pictures are not clickable.
 +
|scope="row"  style=" text-align: left;"| Make the pictures been able to click.
 +
 +
|-
 +
! scope="col"  width="550" style="background:blue; color:#ffffff"|Left-side bar is seldom used by the users. They tend to go back to home page and click on the modules or top bar
 +
! scope="col" width="550" style="background:blue; color:#ffffff"|Together with the quantitative results and statistical analysis, it is highly suggested to keep the top tool bar instead of the side tool bar
 +
 +
 +
|}
 +
 
====Comparing with UT2====
 
====Comparing with UT2====
 
<gallery widths=450px heights=350px perrow=3 mode ="packed">
 
<gallery widths=450px heights=350px perrow=3 mode ="packed">
Line 203: Line 303:
 
  Image:IS480 LittleTeam UT3vsUT2 22.png
 
  Image:IS480 LittleTeam UT3vsUT2 22.png
 
  Image:IS480 LittleTeam UT3vsUT2 3.png
 
  Image:IS480 LittleTeam UT3vsUT2 3.png
 +
</gallery>
 +
 +
=='''User Testing 4'''==
 +
<div align=left style="padding-right:50px; padding-left:50px;">
 +
<font size=3 color="000099">'''Date''': 4 Apr 2014 <br/>
 +
'''Location''': Sengkang Public Library
 +
<br/>'''Participant Group''': Primary School P5 students
 +
<br/>'''Participant Size''': 13;
 +
</font>
 +
</div>
 +
===Testing Documentations===
 +
{| class="wikitable" style="text-align: center; height:50px"
 +
|+
 +
|-
 +
! scope="col"  width="250" style="background:#FFCC33; color:#333333"| Goals&Tasks&Data
 +
! scope="col"  width="250" style="background:#FFCC33; color:#333333"| Scripts
 +
! scope="col"  width="250" style="background:#FFCC33; color:#333333"| Instructions - Set 1
 +
! scope="col"  width="250" style="background:#FFCC33; color:#333333"| Instructions - Set 2
 +
! scope="col"  width="250" style="background:#FFCC33; color:#333333"| Observation Forms
 +
|-
 +
 +
|-
 +
|[[Media:LittleTeam - UT 4 - goals & data & tasks.docx ‎‎|Download]]
 +
|[[Media:LittleTeam - UT 4 - Testing Script.docx ‎ |Download]]‎
 +
|[[Media:LittleTeam - UT 4 - Instructions&feedback - Set1.docx ‎ |Download]]
 +
|[[Media:LittleTeam - UT 4 - Instructions&feedback - Set1.docx ‎ |Download]]
 +
|[[Media:LittleTeam - UT 4 - observation form.xlsx|Download]]
 +
|-
 +
|}
 +
 +
===Testing in Actions===
 +
<gallery widths=350px heights=250px perrow=3 mode ="packed">
 +
LittlteTeam UT4 - photo1.JPG
 +
LittleTeam UT4 -photo2.JPG
 +
 +
</gallery>
 +
 +
===Testing Results===
 +
====Quantitative Results====
 +
<gallery widths=450px heights=350px perrow=3 mode ="packed">
 +
LittleTeam- UT4- charts2.png ‎
 +
LittleTeam- UT4- charts3.png ‎
 +
LittleTeam- UT4- charts4.png ‎
 +
LittleTeam- UT4- charts5.png ‎
 +
LittleTeam- UT4- charts6.png ‎
 +
LittleTeam- UT4- charts7.png
 +
</gallery>
 +
 +
====Qualitative Results====
 +
{| class="wikitable" style="text-align: center; height:50px"
 +
|+
 +
|-
 +
! scope="col"  width="550" style="background:#006633; color:#ffffff"|Top Problems Observed
 +
! scope="col" width="550" style="background:#006633; color:#ffffff"|Improvements 
 +
|-
 +
 +
|scope="row"  style=" text-align: left;"| When creating a lesson in Little Classroom, the different colours of buttons are distracting for the users. It took them some time to figure out the functionalities of different buttons
 +
|scope="row"  style=" text-align: left;"| Simplify and standardize the buttons in Little Classroom.
 +
 +
|-
 +
|scope="row"  style=" text-align: left;"| When the user is in the specific lesson page, the tab of the browser shows "Knowledge Challenge" instead.
 +
|scope="row"  style=" text-align: left;"| Resolve the minor aesthetic bug.
 +
 +
|}
 +
 +
====Comparing with UT3====
 +
<gallery widths=450px heights=350px perrow=3 mode ="packed">
 +
LittleTeam- UT4- charts10.png
 +
LittleTeam- UT4- charts11.png
 +
LittleTeam- UT4- charts12.png
 +
</gallery>
 +
 +
====Comparing with Previous UTs====
 +
<gallery widths=450px heights=350px perrow=3 mode ="packed">
 +
LittleTeam- UT4- charts8.png
 +
LittleTeam- UT4- charts9.png
 +
LittleTeam- UT4- charts1.png
 
</gallery>
 
</gallery>
  
Line 254: Line 431:
 
|-
 
|-
  
 +
|-
 +
!scope="row" width="300" style=" text-align: center;"| Sprint 9
 +
|[[Media:IS480 LittleTeam TestCases Sprint9.xlsx|Download]]
 +
|-
 +
 +
|-
 +
!scope="row" width="300" style=" text-align: center;"| Sprint 10
 +
|[[Media:IS480 LittleTeam TestCases Sprint10.xlsx|Download]]
 +
|-
 +
 +
|-
 +
!scope="row" width="300" style=" text-align: center;"| Sprint 11
 +
|[[Media:LittleTeam TestCases Sprint11.xlsx|Download]]
 +
|-
 +
 +
|-
 +
!scope="row" width="300" style=" text-align: center;"| Sprint 12
 +
|[[Media:LittleTeam TestCases Sprint12.xlsx|Download]]
 +
|-
 
|}
 
|}

Latest revision as of 16:24, 14 April 2014

LittleTeam Logo.jpg
Home The Team Project Overview Project Management Project Documentation Team Resources Team Reflection


UI/UX Design System Design Testing Administrative

Persona

Primary Persona

IS480 LittleTeam PrimaryPersona.JPG

Secondary Persona

IS480 LittleTeam SecondaryPersona.JPG

User Testing 1

Date: 20 Dec 2013
Location: LittleLives
Participant Group: LittleLives Staff
Participant Size: 8

Testing Documentations

Goals&Tasks&Data Scripts Instructions Observation Forms Questionnaires
Download Download Download Download Link

Testing In Actions

Testing Results

Quantitative Data - Time taken to finish the task
IS480 LittleTeam UT1 -r1.png
Quantitative Data - No. of clicks to finish the task
IS480 LittleTeam UT1 -r2.png
Qualitative Data - Usability Rating
IS480 LittleTeam UT1 -r3.png
Frequently Seen Comments Potential Problems & Improvements
"I could not find where to view my score" The score display is not available and the score board design is not properly aligned.
"The success messages are not obvious enough for the users" Change the font size or color of success messages.
"Accept and Reject buttons are not readable" Change the font size and color of the buttons
"The rules for knowledge are not easy and hard to remember" Will implement 'Help' in later sprint when major functions are done
"The navigation bar in Knowledge Challenge takes up too many spaces and is not professional" Change to hidden navigation bar at bottom
"There are too many unnecessary information in Scoreboard, which makes the overall design very crowded" Limited the information displayed in Scoreboard and shift it to the left of the screen
"I cannot view my status of challenges that I answered previously" Create a new page, which displays "Challenge Status"

User Testing 2

Date: 18 Jan 2014 Location: The Learning Lab (United Square)
Participant Group: Secondary Grade 2 - 4 Students
Participant Size: 12;

Testing Documentations

Goals&Tasks&Data Scripts Instructions - Set 1 Instructions - Set 2 Observation Forms
Download Download Download Download Download

Testing In Actions

Testing Results

Quantitative Results

Qualitative Results

Top Comments Potential Problems & Improvements
"The 3 horizontal lines don’t really clearly inform me of its purpose" Remove the horizontal lines and standardize the menu format
"There should be some instructions on choosing the questions for the knowledge challenge" Add in step-by-step guides for users when sending the challenge
"The landing page of Knowledge challenge is too blank and confusing for the first time user" Change the layout of the landing page of knowledge challenge.
"Very hard to find where to search friend" Standardize the top tool bar and merge the search function in.
Generally the system is hard to use for the first time user. Quite a few unnecessary pages and icons. UI Revamp seems necessary, but need to further discuss with client and supervisor

Comparing with UT1

User Testing 3

Date: 15 Feb 2014 Location: The Learning Lab (United Square)
Participant Group: Primary 5 - 6 Students
Participant Size: 12;
Testing Method: A/B Testing. This to...

  • See the user preference on the side/top tool bar
  • Test the difference between these two options statistically.


2 systems are identical except for...

  • System A has only the side tool bar. Refer to here.
  • System B has only the top tool bar. Refer to here.

Testing Documentations

Goals&Tasks&Data Scripts Instructions Observation Forms
Download Download Download Download

Testing in Actions

Testing Results

Quantitative Results

T-test Analysis

Since two different groups of participants test our System A and System B, therefore we ran the statistical standard T-Test for between subjects.

  • Test: Two-tailed t-test at 95% significance level (α=5%), 2 Sample assuming equal variance
  • Independent Variable: System A vs. System B
  • Dependent Variable: Mean time taken to complete the Ask Question task
    • Let μ0 be the mean time to complete the task using System A(control).
    • Let μ1 be the mean time to complete the task using System B(treatment).


  • Hypotheses:
    • Null Hypothesis h0: μ0=μ1
    • Alternative Hypothesis: h1: μ0≠μ1


  • P-value:
    • As calculated using Excel formula: T.Test(array1,array2,2,3), p≈0.02=2%
    • p < α, then significant, not by chance.
    • Therefore we do reject H0.


  • Conclusion: H0 is not true and thus the mean time taken in completing the tasks is significantly different, not by chance.

Qualitative Results

Top Problems Observed Improvements
Calculation of the percentage of voting shows 101%, 1% which are not logically correct Change the calculation methods in back-end.
When sending collaborative challenge, students tend to select 3 (same as sending individual challenge) instead of 1. The reminding note is not obvious, make the notes more obvious to users
When sending challenge to individual person, user tend to fill in all the fields which are prepared for sending challenge to group only. change the forms format to different tabs format to differentiate"To Another User"/"To another Class"/"To another School"
The categories pictures are not clickable. Make the pictures been able to click.
Left-side bar is seldom used by the users. They tend to go back to home page and click on the modules or top bar Together with the quantitative results and statistical analysis, it is highly suggested to keep the top tool bar instead of the side tool bar


Comparing with UT2

User Testing 4

Date: 4 Apr 2014
Location: Sengkang Public Library
Participant Group: Primary School P5 students
Participant Size: 13;

Testing Documentations

Goals&Tasks&Data Scripts Instructions - Set 1 Instructions - Set 2 Observation Forms
Download Download Download Download Download

Testing in Actions

Testing Results

Quantitative Results

Qualitative Results

Top Problems Observed Improvements
When creating a lesson in Little Classroom, the different colours of buttons are distracting for the users. It took them some time to figure out the functionalities of different buttons Simplify and standardize the buttons in Little Classroom.
When the user is in the specific lesson page, the tab of the browser shows "Knowledge Challenge" instead. Resolve the minor aesthetic bug.

Comparing with UT3

Comparing with Previous UTs

Internal Testing

Sprint Test Cases
Sprint 1 Download
Sprint 2 Download
Sprint 3 Download
Sprint 4 Download
Sprint 5 Download
Sprint 6 Download
Sprint 7 Download
Sprint 8 Download
Sprint 9 Download
Sprint 10 Download
Sprint 11 Download
Sprint 12 Download