HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2012T1 6-bit Project Management UT2"

From IS480
Jump to navigation Jump to search
 
(18 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
<div class="center" style="width:60%; margin-left:auto; margin-right:auto;">'''6-bit's Chapalang!''' is a social utility that connects people with friends and new friends <br> by offering a place for exchanging ideas and information on its public domain. <br> http://www.chapalang.com
 
<div class="center" style="width:60%; margin-left:auto; margin-right:auto;">'''6-bit's Chapalang!''' is a social utility that connects people with friends and new friends <br> by offering a place for exchanging ideas and information on its public domain. <br> http://www.chapalang.com
 
</div>
 
</div>
<font face="Arial" size="3">
+
<font face="Calibri">
 
<!--Navigation-->
 
<!--Navigation-->
 
{| style="background-color:#ffffff; color:#000000" width="100%" cellspacing="0" cellpadding="8" valign="top" border="1" |
 
{| style="background-color:#ffffff; color:#000000" width="100%" cellspacing="0" cellpadding="8" valign="top" border="1" |
  
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:90%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Final_Wikipage | <font color="#FFFFFF"><b>Final Wikipage</b></font>]]
+
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:110%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Final_Wikipage | <font color="#FFFFFF"><b>Final Wikipage</b></font>]]
  
 
|}
 
|}
 
{| style="background-color:#ffffff; color:#000000" width="100%" cellspacing="0" cellpadding="8" valign="top" border="0" |
 
{| style="background-color:#ffffff; color:#000000" width="100%" cellspacing="0" cellpadding="8" valign="top" border="0" |
  
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:90%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit | <font color="#FFFFFF"><b>Home</b></font>]]
+
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:110%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit | <font color="#FFFFFF"><b>Home</b></font>]]
  
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:90%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Technical_Overview | <font color="#FFFFFF"><b>Technical Overview</b></font>]]
+
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:110%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Technical_Overview | <font color="#FFFFFF"><b>Technical Overview</b></font>]]
  
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:90%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Project_Deliverables | <font color="#FFFFFF"><b>Project Deliverables</b></font>]]
+
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:110%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Project_Deliverables | <font color="#FFFFFF"><b>Project Deliverables</b></font>]]
  
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:90%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Project_Management | <font color="#FFFFFF"><b>Project Management</b></font>]]
+
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:110%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Project_Management | <font color="#FFFFFF"><b>Project Management</b></font>]]
  
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:90%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Learning_Outcomes | <font color="#FFFFFF"><b>Learning Outcomes</b></font>]]
+
| style="filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#FF0066', endColorstr='#FF0066'); background: -webkit-gradient(linear, left top, left bottom, from(#FF0066), to(#FF0066)); background: -moz-linear-gradient(top,  #FF0066,  #FF0066); font-size:110%; text-align:center; color:#ffffff" width="10%" | [[IS480_Team_wiki:_2012T1_6-bit_Learning_Outcomes | <font color="#FFFFFF"><b>Learning Outcomes</b></font>]]
  
 
|}
 
|}
Line 27: Line 27:
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Schedule</font></div>=
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Schedule</font></div>=
 
==Planned Schedule==
 
==Planned Schedule==
[[Image:6-bit_ScheduleDiagramOverview.png|left|900px|link=""]]
+
[[Image:6-bit_ScheduleDiagramOverview.png|left|750px|link=""]]
 
<br>
 
<br>
 
<br>
 
<br>
Line 86: Line 86:
 
|[[https://docs.google.com/document/d/1Qx4jdRZoE8hXv3u1R4dWke5b-kyMxIc7ASzPIh4ShRQ/edit |<font color="#CD004E"><b>Meeting Minute 6</b></font>]]
 
|[[https://docs.google.com/document/d/1Qx4jdRZoE8hXv3u1R4dWke5b-kyMxIc7ASzPIh4ShRQ/edit |<font color="#CD004E"><b>Meeting Minute 6</b></font>]]
 
|[[https://docs.google.com/document/d/1Pcd6ZADI7ZEuXzq3exrss4nJQhotxAq7-ENfy_OVs6o/edit |<font color="#CD004E"><b>Meeting Minute 16</b></font>]]
 
|[[https://docs.google.com/document/d/1Pcd6ZADI7ZEuXzq3exrss4nJQhotxAq7-ENfy_OVs6o/edit |<font color="#CD004E"><b>Meeting Minute 16</b></font>]]
 +
|[[https://docs.google.com/document/d/1NDCoF37ckdhbys57BLEXShcAcraxzOKpoNbFvpWWUQM/edit |<font color="#CD004E"><b>Meeting Minute 26</b></font>]]
 
|-
 
|-
 
|[[https://docs.google.com/document/d/1uSkv_cux0PqnRTKS-w72wQLgL1ZKaCpXhzDm5uamfMU/edit |<font color="#CD004E"><b>Meeting Minute 7</b></font>]]
 
|[[https://docs.google.com/document/d/1uSkv_cux0PqnRTKS-w72wQLgL1ZKaCpXhzDm5uamfMU/edit |<font color="#CD004E"><b>Meeting Minute 7</b></font>]]
 
|[[https://docs.google.com/document/d/12jW-ycBkfivYL2pg7nrVvb9SF7_XmiglVGGnWAjp6uQ/edit |<font color="#CD004E"><b>Meeting Minute 17</b></font>]]
 
|[[https://docs.google.com/document/d/12jW-ycBkfivYL2pg7nrVvb9SF7_XmiglVGGnWAjp6uQ/edit |<font color="#CD004E"><b>Meeting Minute 17</b></font>]]
 +
|[[https://docs.google.com/document/d/1Amo6YgCkVA3YD-iqyIxhyG8embuClo50KXsznVgoWQE/edit |<font color="#CD004E"><b>Meeting Minute 27</b></font>]]
 
|-
 
|-
 
|[[https://docs.google.com/document/d/1pTivyZ9D3jSU1Kf5VoW4-Xl_C9Abg-fL_6SI6GP_hz0/edit |<font color="#CD004E"><b>Meeting Minute 8</b></font>]]
 
|[[https://docs.google.com/document/d/1pTivyZ9D3jSU1Kf5VoW4-Xl_C9Abg-fL_6SI6GP_hz0/edit |<font color="#CD004E"><b>Meeting Minute 8</b></font>]]
Line 160: Line 162:
 
There is no significant difference between testers’ background in User Test 1 and 2.
 
There is no significant difference between testers’ background in User Test 1 and 2.
 
<br>
 
<br>
[[Image:6bituser-testing2results.png]]
+
[[Image:6bituser-testing2results.png|center]]
  
 
====Test Groups====
 
====Test Groups====
There is no test grouping employed in this test.
+
There are 2 test groups employed in this test.
 +
<br><br>
 +
Group A (Control Group) consists of 17 testers who were testers from User Test 1. The purpose is to test if their return experience with the system has any improvements.
 +
<br>
 +
Group B (Test Group) consists of 29 new testers who have not participated in any of our previous user tests. The purpose of their participation is for comparison, to find out if they have a different experience from the returning testers in Group A.
 +
 
 
====Test Procedures====
 
====Test Procedures====
 
Testers are invited to attend the User Test session and required to bring their own laptops. They are informed on the purpose of the test and given a brief description of the system objective of Chapalang!.  
 
Testers are invited to attend the User Test session and required to bring their own laptops. They are informed on the purpose of the test and given a brief description of the system objective of Chapalang!.  
Subsequently, they are provided with an instruction sheet for a guided test experience. Testers will be required to perform a series of system tasks based on a test case that will go through all the system features and use-case. Thereafter, testers will answer Yes/No binary questions, and can fill in details in an open-ended textbox appended after each question, should they encounter any bugs or suggestions for improvements.
+
<br><br>
While the primary method of testing is on direct user experience and feedback, we have also employed secondary method which collects results indirectly. The number of clicks that each user made, together with the click coordinates or link URL, as well as timestamp of each click is captured for analytical purposes.
+
Subsequently, they are provided with an instruction sheet for a guided test experience. Testers will be required to perform a series of system tasks based on a test case that will go through all the system features and use-case. Thereafter, testers will fill in details in an open-ended textbox appended after each question, should they encounter any bugs or suggestions for improvements.
 +
 
 
====Test Instruction====
 
====Test Instruction====
[https://dl.dropbox.com/u/56071797/User%20Testing%201%20Instructions.docx Click Here to Download User Testing 1 Instruction]
+
[https://dl.dropbox.com/u/56071797/User%20Testing%202%20Instructions.docx Click Here to Download User Testing 2 Instruction]
  
 
====Test Results====
 
====Test Results====
[[Image: 6bituser-testing1resultsfigure.png|250px]]
+
[[Image: 6bituser-testing2resultsfigure.png|250px]]
 
<br>
 
<br>
Based on the abovementioned set of results, it is reasonable to conclude that the system is above satisfactory on functionalities for most testers while there are some rooms for improvements especially on intuitiveness.
+
Based on the survey questions, the results are positive with most testers having a good level of comfort using our web application.
 
<br> <br>
 
<br> <br>
There are also 4 reported bugs, and 19 recommendations for improvements received.  
+
There are also 21 reported bugs, and 69 recommendations for improvements received. The following are the top 10 bugs reported.
 
<br>
 
<br>
[[Image: 6bituser-testing1bug.png|650px]]
+
[[Image: 6bituser-testing2bug.png|600px]]
 
<br>
 
<br>
 
<br>
 
<br>
 
The top 10 most frequently mentioned or important recommendations will be published and appended below.
 
The top 10 most frequently mentioned or important recommendations will be published and appended below.
 
<br>
 
<br>
[[Image: 6bituser-testing1recommendation.png]]
+
[[Image: 6bituser-testing2recommendation.png|600px]]
  
====Click Data Analysis====
+
====Click Data Analysis(User Test 1 vs. User Test 2 – Forum Functions Only)====
Additionally, click data of each test session has also been collected and analysed.  
+
Additionally, click data of each test session has also been collected and analysed. They are also being compared to that with the results of User Test 1.<br>
 +
[[Image: 6bituser-testing2cda1.png]]
 +
<br>
 +
The above box-plot represents 3 sets of data comparing the number of clicks per task, for discussion forum functions only. UT1 represents the results from User Test 1, UT2A represents the results from Group A testers of User Test 2, while UT2B represents the results from Group B testers of User Test 2. For the objective of fair comparisons, the results from User Test 2 has been drilled down to consists of data
 +
<br><br>
 +
The median number of clicks it takes per tester to accomplish a forum-related task in User Test 2 ranges from 1 to 3 clicks with 2 clicks being the median, a decrement from the median of 3 clicks, as well as a smaller variance as compared to User Test 1. Additionally, it can also be observed that there is no significant difference in the results between Group A and Group B users.
 +
<br><br>
 +
Preliminary, we can observe an improvement in the user experience for Group A users between the 2 tests. The improvement can be broadly attributed to the improvements made as well as the high learnability of the system interface design. However, this observation is not conclusive and more data is required.
 
<br>
 
<br>
[[Image: 6bituser-testing1cda1.png]]
+
[[Image: 6bituser-testing2cda2.png]]
 
<br>
 
<br>
Based on the computed statistics illustrated in the box-plot above, the median number of clicks it takes per tester to accomplish a task ranges from 1 to 8 clicks, with 3 clicks being the median. For the purpose of understanding, we could take the measurement of 3 clicks per task as a benchmark, to be compared on subsequent test sessions and observe if there are any improvements.
+
Additional statistics were computer and observed that the median time spent to accomplish a forum-related task for Group A tester is 4 seconds, and Group B tester is 5 seconds. Again, this is a significant decrement in the time spent from User Test 1, where testers spent a median of 10 seconds between each task.  
 +
Based on the finding, we can reasonably derive that there is an improved user experience between User Test 1 and User Test 2, attributing to the improvements made and high learnability of the system. In addition, the improved user experience is shared between both Group A and Group B users, possibly suggesting that the improved system does not require much training or high learning curve.
 +
====Click Data Analysis (Group A vs. Group B – Marketplace Functions)====
 +
Prospectively, we will also study the user experience difference between Group A and Group B testers on marketplace functions, based on the click data which measures the number of clicks involved per task and time taken in seconds between each task.
 
<br><br>
 
<br><br>
[[Image: 6bituser-testing1cda2.png]]
+
In the box-plot diagram above, UT2A refers to User Test 2 Group A testers, while UT2B refers to User Test 2 Group B testers. Each box-plot is represented by data of a specific group of users, and the results computed based on the number of clicks of time pertaining to forum or marketplace functions.
 +
<br>
 +
[[Image: 6bituser-testing2cda3.png]]
 +
<br>
 +
Comparing marketplace functions, both Group A and Group B testers have made a median of 2 clicks to accomplish each task. While Group B testers have a wider variance of clicks of up to 4 clicks, it can be broadly attributed to outliers, user experiments or some learning curve involved in getting used to the functions or interface objects placements.
 
<br>
 
<br>
More statistics were computed in the box-plot diagram above to understand the median time spent to accomplish a task and we observe that the time taken ranges from 4 seconds to 18 seconds, with 10 seconds being the median. We will also be taking the measurement of 10 seconds per task as a benchmark to compare with subsequent test sessions and observe if there are any improvements.
+
[[Image: 6bituser-testing2cda4.png]]
 
+
<br>
====Additional Observations====
+
The result when comparing the time taken is consistent with the preliminary conclusion when comparing the number of clicks per task. The median time taken for Group A and Group B testers for forum and marketplace functions are within the range of 4 to 5 seconds. The difference between the median records is insignificant.
Further drill down on the click data has identified an observation where most testers take more clicks to accomplish Step 4 of the instructions. In Step 4, testers are required to subscribe a forum. In this observation, we are able to relate that testers are unable to easily find the Subscribe to Forum button. This finding is aligned with a common feedback that we should consider replicating the subscription button to a more intuitive location on the webpage.
+
<br><br>
 +
Overall, the result is consistent across forums and marketplace functions, between testers from both User Tests and test groups. It is also consistent with our earlier preliminary conclusion that the improvements made between the two User Tests have resulted in improved user experience, and there is a good level of learnability in the interface design.
 +
<br><br>
 +
While there are limitations in this test, where there are other externalities such as network performance, computing habits of testers and response time of each users, the macro results of the test provide a reasonable sampling on the objective of the test.
  
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Milestones</font></div>=
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Milestones</font></div>=
  
[[Image:6-bit_milestones.png|750px]]
+
[[Image:6-bit_schedule.png|600px|center]]
  
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Schedule Metric</font></div>=
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Schedule Metric</font></div>=
<br>
+
Every iteration, schedule metric values are calculated to understand the project progress. They are broadly categorized into 5 different groups, where different action plans will apply. The acceptable range of value is within 90% to 110%, offering some buffer for natural inaccuracies between forecasting and execution.
[[Image:6-bit_ScheduleMetric.png|750px]]
+
<br><br>
<br>
+
Total Schedule Metric Value = Planned no. of days taken (P) / Actual no. of Days Assigned (A) x 100%
<br>
+
<br><br>
 +
[[Image:6-bit_schedulemetric.png|center|600px]]
  
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Bug Metric</font></div>=
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Bug Metric</font></div>=
 +
==Log==
 +
[[Image:6-bit_BugMetric.png|550px]]
 +
[[Image:6-bit_BugLog.png|600px]]
 +
<div style="text-align: right;">
 +
Bug Log: [https://docs.google.com/spreadsheet/ccc?key=0Aqd6IiSLbMwQdEI5TldNSEhpcVRjb1puYzU3ZHJOckE |<font color="#CD004E"><b>Click Here</b></font>]
 +
</div>
 
<br>
 
<br>
[[Image:6-bit_BugMetric.png|750px]]
+
Bug logging for Chapalang! takes the direction of being practical and easily monitored from both macro and micro perspectives. Whenever a bug is found, a new row is entered with the following data:
Bug Log: [https://docs.google.com/spreadsheet/ccc?key=0Aqd6IiSLbMwQdEI5TldNSEhpcVRjb1puYzU3ZHJOckE |<font color="#CD004E"><b>Click Here</b></font>]
+
* Index number
 +
* Bug description
 +
* Found by
 +
* Found date
 +
* Expected solve-by date
 +
* Bug severity
 +
* Status
 +
* Owner of the function
 +
* Fixed date
 +
* Closed by (Tester)
 +
* Close date
 +
* Additional comments
 +
==Metric==
 +
Bugs are classified into 3 different categories of complexity, easy, moderate and hard. Each category is assigned points of 1, 5 and 10 respectively, lower is better.
 +
<br><br>
 +
Total Points for Each Iteration = Σ Points of the Bugs in each iteration
 +
<br>
 +
[[Image:6-bit_BugMetricFormula.png|500px|center]]
 
<br>
 
<br>
 +
After assigning each bug with points associated by its complexity, we will track the total bug scores at the end of each week before deciding if there should be any actions to be taken. The following is an action plan for our bug metric:
 
<br>
 
<br>
 +
[[Image:6-bit_BugMetricFormula2.png|500px|center]]
  
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Risk & Mitigation</font></div>=
 
=<div style="background: #FF0080; background: -webkit-gradient(linear, left top, left bottom, from(#FF0080), to(#F660AB)); padding: 12px; font-weight: bold; text-align: center "><font color="white" size="6" >Risk & Mitigation</font></div>=
 
<br>
 
<br>
[[Image:6-bit_RiskDiagram.png|750px]]
+
[[Image:6-bit_RiskDiagram.png|600px]]
 
<br>
 
<br>
 
<br>
 
<br>

Latest revision as of 11:44, 4 December 2012

6-bit logo.png
6-bit's Chapalang! is a social utility that connects people with friends and new friends
by offering a place for exchanging ideas and information on its public domain.
http://www.chapalang.com

Final Wikipage
Home Technical Overview Project Deliverables Project Management Learning Outcomes


Schedule

Planned Schedule

6-bit ScheduleDiagramOverview.png

































Meeting Minutes

Team Meeting Minutes

[|Meeting Minute 1] [|Meeting Minute 11] [|Meeting Minute 21]
[|Meeting Minute 2] [|Meeting Minute 12] [|Meeting Minute 22]
[|Meeting Minute 3] [|Meeting Minute 13] [|Meeting Minute 23]
[|Meeting Minute 4] [|Meeting Minute 14] [|Meeting Minute 24]
[|Meeting Minute 5] [|Meeting Minute 15] [|Meeting Minute 25]
[|Meeting Minute 6] [|Meeting Minute 16] [|Meeting Minute 26]
[|Meeting Minute 7] [|Meeting Minute 17] [|Meeting Minute 27]
[|Meeting Minute 8] [|Meeting Minute 18]
[|Meeting Minute 9] [|Meeting Minute 19]
[|Meeting Minute 10] [|Meeting Minute 20]

Supervisor Meeting Minutes

|Meeting Minute 1
|Meeting Minute 2
|Meeting Minute 3
|Meeting Minute 4
|Meeting Minute 5
|Meeting Minute 6
|Meeting Minute 7
|Meeting Minute 8
|Meeting Minute 9
|Meeting Minute 10
|Meeting Minute 11

Testing

Test Cases

|Test Cases

Test Plans

Test Plan 1 on 17 September 2012
Test Plan 2 on 28 September 2012
Test Plan 3 on 19 October 2012
Test Plan 4 on 4 November 2012

User Testing

User Testing 1 User Testing 2 User Testing 3 User Testing 4

User Testing 2

6bituser-testing2.png

Test Description

The objective of User Test 2 is on functionality and usability testing of the system. The coverage of the test is focused on forum and basic marketplace functions.

The purpose of re-testing on forum function is to test for an experience improvement. Basic marketplace functions will consist of a complete product display to payment process. We will want to find out if the changes made after User Test 1 is offering an improved user experience to users, and whether the improvement is consistent between forum and marketplace functions.

Testers Background

There are a total of 46 testers who attended the User Test, of which 61% (28) are male and 39% (18) are female, representing various schools in SMU with SIS students being the majority. It is also observed that most testers are users of Chrome and Firefox web browser. There is no significant difference between testers’ background in User Test 1 and 2.

6bituser-testing2results.png

Test Groups

There are 2 test groups employed in this test.

Group A (Control Group) consists of 17 testers who were testers from User Test 1. The purpose is to test if their return experience with the system has any improvements.
Group B (Test Group) consists of 29 new testers who have not participated in any of our previous user tests. The purpose of their participation is for comparison, to find out if they have a different experience from the returning testers in Group A.

Test Procedures

Testers are invited to attend the User Test session and required to bring their own laptops. They are informed on the purpose of the test and given a brief description of the system objective of Chapalang!.

Subsequently, they are provided with an instruction sheet for a guided test experience. Testers will be required to perform a series of system tasks based on a test case that will go through all the system features and use-case. Thereafter, testers will fill in details in an open-ended textbox appended after each question, should they encounter any bugs or suggestions for improvements.

Test Instruction

Click Here to Download User Testing 2 Instruction

Test Results

6bituser-testing2resultsfigure.png
Based on the survey questions, the results are positive with most testers having a good level of comfort using our web application.

There are also 21 reported bugs, and 69 recommendations for improvements received. The following are the top 10 bugs reported.
6bituser-testing2bug.png

The top 10 most frequently mentioned or important recommendations will be published and appended below.
6bituser-testing2recommendation.png

Click Data Analysis(User Test 1 vs. User Test 2 – Forum Functions Only)

Additionally, click data of each test session has also been collected and analysed. They are also being compared to that with the results of User Test 1.
6bituser-testing2cda1.png
The above box-plot represents 3 sets of data comparing the number of clicks per task, for discussion forum functions only. UT1 represents the results from User Test 1, UT2A represents the results from Group A testers of User Test 2, while UT2B represents the results from Group B testers of User Test 2. For the objective of fair comparisons, the results from User Test 2 has been drilled down to consists of data

The median number of clicks it takes per tester to accomplish a forum-related task in User Test 2 ranges from 1 to 3 clicks with 2 clicks being the median, a decrement from the median of 3 clicks, as well as a smaller variance as compared to User Test 1. Additionally, it can also be observed that there is no significant difference in the results between Group A and Group B users.

Preliminary, we can observe an improvement in the user experience for Group A users between the 2 tests. The improvement can be broadly attributed to the improvements made as well as the high learnability of the system interface design. However, this observation is not conclusive and more data is required.
6bituser-testing2cda2.png
Additional statistics were computer and observed that the median time spent to accomplish a forum-related task for Group A tester is 4 seconds, and Group B tester is 5 seconds. Again, this is a significant decrement in the time spent from User Test 1, where testers spent a median of 10 seconds between each task. Based on the finding, we can reasonably derive that there is an improved user experience between User Test 1 and User Test 2, attributing to the improvements made and high learnability of the system. In addition, the improved user experience is shared between both Group A and Group B users, possibly suggesting that the improved system does not require much training or high learning curve.

Click Data Analysis (Group A vs. Group B – Marketplace Functions)

Prospectively, we will also study the user experience difference between Group A and Group B testers on marketplace functions, based on the click data which measures the number of clicks involved per task and time taken in seconds between each task.

In the box-plot diagram above, UT2A refers to User Test 2 Group A testers, while UT2B refers to User Test 2 Group B testers. Each box-plot is represented by data of a specific group of users, and the results computed based on the number of clicks of time pertaining to forum or marketplace functions.
6bituser-testing2cda3.png
Comparing marketplace functions, both Group A and Group B testers have made a median of 2 clicks to accomplish each task. While Group B testers have a wider variance of clicks of up to 4 clicks, it can be broadly attributed to outliers, user experiments or some learning curve involved in getting used to the functions or interface objects placements.
6bituser-testing2cda4.png
The result when comparing the time taken is consistent with the preliminary conclusion when comparing the number of clicks per task. The median time taken for Group A and Group B testers for forum and marketplace functions are within the range of 4 to 5 seconds. The difference between the median records is insignificant.

Overall, the result is consistent across forums and marketplace functions, between testers from both User Tests and test groups. It is also consistent with our earlier preliminary conclusion that the improvements made between the two User Tests have resulted in improved user experience, and there is a good level of learnability in the interface design.

While there are limitations in this test, where there are other externalities such as network performance, computing habits of testers and response time of each users, the macro results of the test provide a reasonable sampling on the objective of the test.

Milestones

6-bit schedule.png

Schedule Metric

Every iteration, schedule metric values are calculated to understand the project progress. They are broadly categorized into 5 different groups, where different action plans will apply. The acceptable range of value is within 90% to 110%, offering some buffer for natural inaccuracies between forecasting and execution.

Total Schedule Metric Value = Planned no. of days taken (P) / Actual no. of Days Assigned (A) x 100%

6-bit schedulemetric.png

Bug Metric

Log

6-bit BugMetric.png 6-bit BugLog.png

Bug Log: |Click Here


Bug logging for Chapalang! takes the direction of being practical and easily monitored from both macro and micro perspectives. Whenever a bug is found, a new row is entered with the following data:

  • Index number
  • Bug description
  • Found by
  • Found date
  • Expected solve-by date
  • Bug severity
  • Status
  • Owner of the function
  • Fixed date
  • Closed by (Tester)
  • Close date
  • Additional comments

Metric

Bugs are classified into 3 different categories of complexity, easy, moderate and hard. Each category is assigned points of 1, 5 and 10 respectively, lower is better.

Total Points for Each Iteration = Σ Points of the Bugs in each iteration

6-bit BugMetricFormula.png


After assigning each bug with points associated by its complexity, we will track the total bug scores at the end of each week before deciding if there should be any actions to be taken. The following is an action plan for our bug metric:

6-bit BugMetricFormula2.png

Risk & Mitigation


6-bit RiskDiagram.png