HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2013T2 GENShYFT Documentation User"

From IS480
Jump to navigation Jump to search
 
(35 intermediate revisions by 2 users not shown)
Line 41: Line 41:
 
| style="padding:0.4em; font-size:150%; background-color:#FFFFFF;  border-bottom:4px solid #2f2929; border-top:5px solid #2f2929; text-align:center; color:#828282" width="10%" | [[IS480_Team_wiki:_2013T2_GENShYFT_Documentation_proto  | <font color="#000000" size=2><b>Prototype</b></font>]]
 
| style="padding:0.4em; font-size:150%; background-color:#FFFFFF;  border-bottom:4px solid #2f2929; border-top:5px solid #2f2929; text-align:center; color:#828282" width="10%" | [[IS480_Team_wiki:_2013T2_GENShYFT_Documentation_proto  | <font color="#000000" size=2><b>Prototype</b></font>]]
  
|}
+
|}<br />
 +
<br />
 +
=Regression Test=
 +
Adhering to the advice given by reviewers during Mid Term Review, we decided to write scripts to automate Regression Testing. We approached this by leveraging on the Automated Testing capabilities of Singpath and further extended it.<br /><br />
 +
 
 +
Although there are many different ways to implement the scripts for automated testing, we have decided on the one that is the most cost-effective in terms of resources required - E2E test<br />
 +
<br />
 +
 
 +
'''Automated E2E Tests'''<br />
 +
As our features in Singpath is already on the Live Server since Mid Terms, our development work may affect the existing codes and the most obvious indication that Singpath broke is not so much on the logic, but the view/display layer (User Interface). <br /><br />
 +
To the end users, the pages will simply not load/show if there is something wrong with our code, to address that issue, we have specifically zeroed in to write additional test scripts to cover all of our pages.<br />
 +
<br />
 +
'''How it works'''<br />
 +
 
 +
Test scripts are written into a file within the application, and will be executed when it is sent to Travis CI, a distributed continuous integration service used to build and test projects hosted at GitHub, which is the repository that we have been using for the development of Singpath. <br />
 +
 
 +
Travis CI automatically detects when a commit has been made and pushed to a GitHub repository that is using Travis CI, and each time this happens, it will try to build the project and run tests.  Travis CI will also build and run pull requests. When that process has completed, it will notify our Client as well as showing the test results (success or failure) with our pull request (pre-deployment check). <br />
 +
 
 +
What this means is that every time we commit and send a pull request, the test script is run automatically and we will be shown a result whether something in our new codes 'broke' the application. With this, we are confident to say that every time we push new changes to the live site, the view/display layer will always render.<br />
 +
 
 +
 
 +
'''Results/Outcome'''<br />
 +
 
 +
Extended routeSpec.js test scripts by 125%, added 26 additional routing to the existing 19.<br />
 +
<br />
 +
Win-win - our live deployment will never be shown 'broken pages', future teams can make use of our scripts (Value Add for Client)
  
 
<br/>
 
<br/>
  
 
+
=User Testing=
{| class="wikitable centered" width="90%"
+
<!--Please click [[Media:GENShYFT Test Plan 140221.pdf|here]] for our Test Plan.<br />-->
!UAT
+
We employed a variety of evaluation techniques learnt from IS306 class the previous semester. The evaluation tests that we administered include the following methods:
 +
# Lab Experiment
 +
# Heuristic Evaluation
 +
# Field Study
 +
# Sample Survey
 +
# Judgement Study
 +
# [http://www.intuitionhq.com/blog/2011/04/ab-and-preference-testing-for-usability Preference Test]
 +
<br />
 +
{| class="wikitable centered" width="100%"
 +
!Method
 
!Details
 
!Details
 
!Documents
 
!Documents
Line 53: Line 87:
  
 
|-
 
|-
|1
+
|Lab Experiment<br />
 +
Heuristic Evaluation
 
|Date: 11/2/2014<br />
 
|Date: 11/2/2014<br />
 
Participants: 15<br />
 
Participants: 15<br />
Location: SMU SIS GSR 2-5
+
Location: SMU SIS GSR 2-5<br />
#REDIRECT [[IS480_Team_wiki:_2013T2_GENShYFT_Documentation_User_UT1_Gallery|Photo Gallery]]
+
Objective: Functionality Test and Heuristic Evaluation<br /><br />
 +
 
 +
<div class="center">
 +
<imagemap>
 +
Image:GENShYFTFirstTest.png|600px|alt=GENShYFT First Test
 +
default [https://wiki.smu.edu.sg/is480/IS480_Team_wiki%3A_2013T2_GENShYFT_Documentation_User_UT1_Gallery]
 +
</imagemap>
 +
[[IS480_Team_wiki:_2013T2_GENShYFT_Documentation_User_UT1_Gallery|Photo Gallery]]
 
|
 
|
[[Media:AY1314-T2_IS480_GENShYFT_Tester's Package A.pdf|Tester's Package A (Detailed)]]<br />
+
[[Media:AY1314-T2_IS480_GENShYFT_Tester's Package A.pdf|Tester's Package A]]<br />
[[Media:AY1314-T2_IS480_GENShYFT_Tester's Package B.pdf|Tester's Package B (Scenario)]]<br />
+
[[Media:AY1314-T2_IS480_GENShYFT_Tester's Package B.pdf|Tester's Package B]]<br />
 
[[Media:AY1314-T2_IS480_GENShYFT_Script.pdf|Script]]<br />
 
[[Media:AY1314-T2_IS480_GENShYFT_Script.pdf|Script]]<br />
[[Media:GENShYFT Test Plan 140217.pdf|Test Plan]]<br />
 
 
[https://docs.google.com/forms/d/1fTnlxzCZzOGsBypmR7XrH9yanaRQ_RyhJQTWagaDS1Q Heuristic Evaluation]<br />
 
[https://docs.google.com/forms/d/1fTnlxzCZzOGsBypmR7XrH9yanaRQ_RyhJQTWagaDS1Q Heuristic Evaluation]<br />
 
[https://docs.google.com/forms/d/1_HDwRX0Q9GD5Ak3j81jbxKLTrUwCwyzHXEgEs1kosBA Questionnaire]<br />
 
[https://docs.google.com/forms/d/1_HDwRX0Q9GD5Ak3j81jbxKLTrUwCwyzHXEgEs1kosBA Questionnaire]<br />
 
[https://docs.google.com/forms/d/15Y2hghRZeRLCDS8BIrLburykacqsrv6ODEZLlMVSAkY Observer's Guide]<br />
 
[https://docs.google.com/forms/d/15Y2hghRZeRLCDS8BIrLburykacqsrv6ODEZLlMVSAkY Observer's Guide]<br />
 
|
 
|
[[Media:GENShYFT UAT 1 Test Results.pdf|UAT 1 Test Results]]<br />
+
[[Media:GENShYFT UAT 1 Test Results.pdf|UAT 1 & HE 1 Test Results]]<br />
 
|-
 
|-
|2
+
|Field Study<br />
|Date: 1/3/2014<br />
+
|Date: 19/3/2014<br />
Participants: 15 (Expected) <br />
+
'''In-class Tournament Held [Real Life Use Case]'''<br />
Location: SMU SIS GSR 2-5
+
Participants: 30 <br />
|Documents here
+
Location: SMU SIS SR 3-2 (IS429 Week 11 Class)<br />
|
+
Objective: Load Test to ensure Singpath can handle an entire class of concurrent users<br /><br />
 +
 
 +
<div class="center">
 +
<imagemap>
 +
Image:GENShYFTSecondTest.png|600px|alt=GENShYFT Second Test
 +
default [https://www.facebook.com/media/set/?set=a.289817004509197.1073741830.288817147942516&type=1]
 +
</imagemap>
 +
[https://www.facebook.com/media/set/?set=a.289817004509197.1073741830.288817147942516&type=1| View More Pictures here!]
 +
|None
 +
|Observation - Load Test Successful<br />
 +
<br />
 +
[[Media:GENShYFT_Observations.pdf|Observations]]<br />
 
|-
 
|-
|3
+
|Field Study
 
|Date: 21/3/2014<br />
 
|Date: 21/3/2014<br />
Participants: 20 (Expected) <br />
+
'''Actual Tournament Held [Real Life Use Case]''' <br />
Location: SMU SIS SR 2-1
+
Participants: 20 Secondary School students (Actual Users) <br />
|Documents here
+
Location: SMU SIS SR 2-1<br />
|
+
Objective: Field Study to observe users interaction with Singpath in an uncontrolled environment<br />
 +
<div class="center">
 +
<imagemap>
 +
Image:GENShYFTThirdTest.png|600px|alt=GENShYFT Third Test
 +
default [https://www.facebook.com/media/set/?set=a.290625361095028.1073741831.288817147942516&type=1]
 +
</imagemap>
 +
[https://www.facebook.com/media/set/?set=a.290625361095028.1073741831.288817147942516&type=1| View More Pictures here!] Part 1<br />
 +
[https://www.facebook.com/media/set/?set=a.292588620898702.1073741832.288817147942516&type=1| View More Pictures here!] Part 2
 +
|[http://tinyurl.com/spidasecgs Post IDA Sec Sch Tournament Survey]
 +
|Sec Sch Post Tournament Survey<br />
 +
Unable to administer survey; requires clearance by sponsor IDA<br />
 +
Tournament proceeded smoothly with minor hiccups<br />
 +
<br />
 +
[[Media:GENShYFT_Observations.pdf|Observations]]<br />
 
|-
 
|-
|4
+
|Field Study<br />
 +
Judgement Study<br />
 +
Sample Survey
 
|Date: 29/3/2014<br />
 
|Date: 29/3/2014<br />
Participants: 40 (Expected) <br />
+
'''Actual Tournament Held [Real Life Use Case]'''<br />
Location: SMU SIS SR 2-1
+
Participants: 35 JC/High School Students (Actual Users)<br />
|Documents here
+
Location: SMU SIS SR 2-1<br />
|
+
Objective: Field Study to observe users interaction with Singpath in an uncontrolled environment<br /><br />
 +
 
 +
<div class="center">
 +
<imagemap>
 +
Image:GENShYFTFourthTest.png|600px|alt=GENShYFT Fourth Test
 +
default [https://www.facebook.com/media/set/?set=a.293511244139773.1073741834.288817147942516&type=1]
 +
</imagemap>
 +
[https://www.facebook.com/media/set/?set=a.293511244139773.1073741834.288817147942516&type=1| View More Pictures here!]
 +
|[http://tinyurl.com/jcsingpath Post PayPal JC Tournament Survey]
 +
|[[Media:GENShYFT_JCSurveyResults.pdf|JC Post Tournament Survey Results]]<br />
 +
Tournament proceeded smoothly without hiccups<br />
 +
<br />
 +
[[Media:GENShYFT_Observations.pdf|Observations]]<br />
 
|-
 
|-
|5
+
|Field Study<br />
 +
Judgement Study<br />
 +
Sample Survey
 
|Date: 29/3/2014<br />
 
|Date: 29/3/2014<br />
Participants: 40 (Expected) <br />
+
'''Actual Tournament Held [Real Life Use Case]'''<br />
Location: SMU SIS SR 2-1
+
Participants: 29 Poly Students (Actual Users)<br />
|Documents here
+
Location: SMU SIS SR 2-1<br />
|
+
Objective: Field Study to observe users interaction with Singpath in an uncontrolled environment<br /><br />
 +
 
 +
<div class="center">
 +
<imagemap>
 +
Image:GENShYFTFifthTest.png|600px|alt=GENShYFT Fifth Test
 +
default [https://www.facebook.com/media/set/?set=a.293517594139138.1073741835.288817147942516&type=1]
 +
</imagemap>
 +
[https://www.facebook.com/media/set/?set=a.293517594139138.1073741835.288817147942516&type=1| View More Pictures here!]
 +
|[http://tinyurl.com/singpathpoly Post PayPal Poly Tournament Survey]
 +
|[[Media:GENShYFT_PolySurveyResults.pdf|Poly Post Tournament Survey Results]]<br />
 +
Tournament proceeded smoothly without hiccups<br />
 +
<br />
 +
[[Media:GENShYFT_Observations.pdf|Observations]]<br />
 
|-
 
|-
  
  
 
|}
 
|}

Latest revision as of 08:06, 17 April 2014

GENShYFT IS480 1314 Logo.jpg


Home   Project Overview   Project Management   Documentation   The Team


Overview   Diagrams   User Testing   Presentation   Prototype



Regression Test

Adhering to the advice given by reviewers during Mid Term Review, we decided to write scripts to automate Regression Testing. We approached this by leveraging on the Automated Testing capabilities of Singpath and further extended it.

Although there are many different ways to implement the scripts for automated testing, we have decided on the one that is the most cost-effective in terms of resources required - E2E test

Automated E2E Tests
As our features in Singpath is already on the Live Server since Mid Terms, our development work may affect the existing codes and the most obvious indication that Singpath broke is not so much on the logic, but the view/display layer (User Interface).

To the end users, the pages will simply not load/show if there is something wrong with our code, to address that issue, we have specifically zeroed in to write additional test scripts to cover all of our pages.

How it works

Test scripts are written into a file within the application, and will be executed when it is sent to Travis CI, a distributed continuous integration service used to build and test projects hosted at GitHub, which is the repository that we have been using for the development of Singpath.

Travis CI automatically detects when a commit has been made and pushed to a GitHub repository that is using Travis CI, and each time this happens, it will try to build the project and run tests. Travis CI will also build and run pull requests. When that process has completed, it will notify our Client as well as showing the test results (success or failure) with our pull request (pre-deployment check).

What this means is that every time we commit and send a pull request, the test script is run automatically and we will be shown a result whether something in our new codes 'broke' the application. With this, we are confident to say that every time we push new changes to the live site, the view/display layer will always render.


Results/Outcome

Extended routeSpec.js test scripts by 125%, added 26 additional routing to the existing 19.

Win-win - our live deployment will never be shown 'broken pages', future teams can make use of our scripts (Value Add for Client)


User Testing

We employed a variety of evaluation techniques learnt from IS306 class the previous semester. The evaluation tests that we administered include the following methods:

  1. Lab Experiment
  2. Heuristic Evaluation
  3. Field Study
  4. Sample Survey
  5. Judgement Study
  6. Preference Test


Method Details Documents Results
Lab Experiment

Heuristic Evaluation

Date: 11/2/2014

Participants: 15
Location: SMU SIS GSR 2-5
Objective: Functionality Test and Heuristic Evaluation

Tester's Package A
Tester's Package B
Script
Heuristic Evaluation
Questionnaire
Observer's Guide

UAT 1 & HE 1 Test Results

Field Study
Date: 19/3/2014

In-class Tournament Held [Real Life Use Case]
Participants: 30
Location: SMU SIS SR 3-2 (IS429 Week 11 Class)
Objective: Load Test to ensure Singpath can handle an entire class of concurrent users

None Observation - Load Test Successful


Observations

Field Study Date: 21/3/2014

Actual Tournament Held [Real Life Use Case]
Participants: 20 Secondary School students (Actual Users)
Location: SMU SIS SR 2-1
Objective: Field Study to observe users interaction with Singpath in an uncontrolled environment

Post IDA Sec Sch Tournament Survey Sec Sch Post Tournament Survey

Unable to administer survey; requires clearance by sponsor IDA
Tournament proceeded smoothly with minor hiccups

Observations

Field Study

Judgement Study
Sample Survey

Date: 29/3/2014

Actual Tournament Held [Real Life Use Case]
Participants: 35 JC/High School Students (Actual Users)
Location: SMU SIS SR 2-1
Objective: Field Study to observe users interaction with Singpath in an uncontrolled environment

Post PayPal JC Tournament Survey JC Post Tournament Survey Results

Tournament proceeded smoothly without hiccups

Observations

Field Study

Judgement Study
Sample Survey

Date: 29/3/2014

Actual Tournament Held [Real Life Use Case]
Participants: 29 Poly Students (Actual Users)
Location: SMU SIS SR 2-1
Objective: Field Study to observe users interaction with Singpath in an uncontrolled environment

Post PayPal Poly Tournament Survey Poly Post Tournament Survey Results

Tournament proceeded smoothly without hiccups

Observations