HeaderSIS.jpg

IS480 Team wiki: 2012T1 Innox Project Management Testing

From IS480
Jump to navigation Jump to search
INNOX TEAM LOGO.png


Project Management


Innovating Since 2010


Home   The Team / LOMS   Project Overview Project Documentation   Project Management   Resources & References


  Overview   Development   Testing   Design  ]]



Contents

Test Schedule

Version 3

Sprint Milestone Planned Start Date Planned End Date Actual Start Date Actual End Date Remarks
T1 System Integration Test(Queue, Alerts, Services) 21st January 30th January 27th January 30th January Completed
T2 User Test - Client 14th February 20th February 15th February 20th February Completed
T3 User Acceptance Test - Client 8th March 13th March 8th March 13th March Completed
T4 Indoor Positioning System Function Test 25th March 25th March 25th March 25th March Completed
T5 User Acceptance Test - Registrar 10th April 12th April 10th April 12th April Completed

Version 2

Sprint Milestone Planned Start Date Planned End Date Actual Start Date Actual End Date Remarks
T1 System Integration Test(Queue, Alerts, Services) 21st January 30th January 27th January 30th January Completed
T2 User Test - Client 14th February 20th February 15th February 20th February Completed
T3 User Test - Registrars 28th February 14th March
T4 User Test - Private Law Firms 28th March 14th March Cancelled

Version 1

Sprint Milestone Planned Start Date Planned End Date Actual Start Date Actual End Date Remarks
T1 System Integration Test(Queue, Alerts, Services) 21st January 30th January 27th January 30th January Completed
T2 User Test @ Supreme Court 13th February 18th February Cancelled


System Integration Test

Objectives

To test out the functionalities that have been completed thus far in the schedule. The following functionalities were tested:

  • Queue Management System Module
  • Alerts Module
  • Services Module

Details

  • Testers: Rizwan & Wei Min
  • Venue: School of Information Systems
  • Date: 27th - 30th January 2013
  • Duration: 1 week

Method

  • Our Testers had created Test Cases to perform tests on the Mobile Application. The testers were not heavily involved in the development of the functions to avoid any conflicts.
  • These test cases were conducted throughout the week of testing.

Data collection

Collecting of results

  • Bugs from the test was identified and collected in the Bug Defect list
  • These bugs was then assigned to the developers to fix
  • Developers had to fix the bugs and report back to the testers and Project Manager

Mobile Phones Used

These are the phone models used by the team:

  • Samsung S3 GT-I9305, Andriod version 4.1.1
  • LG Optimus 2X, Android 4.1.1
  • Samsung S2 GT-I9100, Andriod 2.3.6

Test Cases

INNOX FUNCTIONALITY TABLE2.png







Test Results

INNOX BUG FUNCTIONALITY GRAPH2.png





















Bug Distribution

INNOX FUNCTIONALITY Distribution1.png

















Detailed Test Cases

Date Document
29 Jan 2013 Functionality Test Case
29 Jan 2013 Functionality Test Results
29 Jan 2013 Defect Tracker

User Test-Client

Objectives

For our client to test out the functionalities that have been completed thus far in the schedule. The following functionalities were tested:

  • Queue Management System Module
  • Alerts Module
  • Hearing List Module
  • Services Module

Details

  • Participants: 4 Court Employees
  • Venue: Vicinity of the Court
  • Date: 15th - 20th February 2013
  • Duration: 1 week

Method

  • We have packaged our project into an .APK file and submitted it to our client to test on.
  • Together with the .APK file, we sent them a copy of the bug defect tracker for them to log and defects found.
  • Cases had to be filled up into the server on a daily basis to ensure testing went through without any glitch.
  • The employees conducted the tests throughout the week.

Data collection

  • Clients filled up the defect tracker and sent it to the team at the end of the week.
  • These bugs was then assigned to the developers to fix
  • Developers had to fix the bugs and report back to the testers and Project Manager

Mobile Phone Models

These are the phone models used by the various Supreme Court Staff:

  • HTC One X+, Andriod 4.1.1
  • Samsung S3 GT-I9305, Andriod version 4.1.1
  • Samsung S3 GT-I9300, Andriod version 4.1.2
  • Samsung S2 GT-I9100, Andriod 2.3.6


Bug Distribution

INNOX CleintTest Bugs2.png

















Deocuments

Date Document
21 Feb 2013 Defect Tracker

User Acceptance Test - Client

Objectives

For our client to test out the functionalities that have been completed thus far in the schedule. The following functionalities were tested:

  • Queue Management System Module
  • Alerts Module
  • Hearing List Module
  • Services Module
  • Admin Module

Details

  • Participants: 5 Court Employees
  • Venue: Vicinity of the Court
  • Date: 08th - 13th March 2013
  • Duration: 6 days

Method

  • We have packaged our project into an .APK file and submitted it to our client to test on.
  • Together with the .APK file, we sent them a copy of the bug defect tracker for them to log and defects found.
  • Cases had to be filled up into the server on a daily basis to ensure testing went through without any glitch.
  • The employees conducted the tests throughout the week.

Data collection

Collecting of results

  • Clients filled up the defect tracker and sent it to the team at the end of every testing day.
  • These bugs was then assigned to the developers to fix
  • Developers had to fix the bugs and report back to the testers and Project Manager

Mobile Phone Models

These are the phone models used by the various Supreme Court Staff:

  • HTC One X+, Andriod 4.1.1
  • Samsung S3 GT-I9305, Andriod version 4.1.1
  • Samsung S3 GT-I9300, Andriod version 4.1.2
  • Samsung S2 GT-I9100, Andriod 2.3.6

Test Cases

INNOX UATClient TABLE.png







Test Results

INNOX BUG UATClient GRAPH.png





















Bug Distribution

INNOX UATClient Distribution.png

















Detailed Test Cases

Date Document
14 Mar 2013 Functionality Test Case
14 Mar 2013 Functionality Test Results
14 Mar 2013 Defect Tracker

Indoor Positioning System Function Test

Objectives

For our client to test out the functionalities that have been completed thus far in the schedule. The following functionalities were tested:

  • Nearest Neighbour Method
  • Probabilistic Method

Details

  • Participants: 2 Innox Student Team Members
  • Venue: Supreme Court
  • Date: 25th March 2013
  • Duration: 1 day

Method

  • Mapped 22 different points in Supreme-Court Level 1 and 2 (~5m apart)
  • Measured Each point 120 times(60times with S3 and 60 with S2)
  • Results logged to determine if the desired point had jumped to another point
  • Average accuracy: 94.09% over a 5m radius

Results

  • Nearest Neighbour gave the best Result compared to Probabilistic Method
  • Nearest Neighbour Accuracy: 94.09% over a 5m radius
  • Probabilistic Method Accuracy: 78.17% over a 5m radius

Nearest Neighbour Accuracy Distribution

INNOX Wifi Accuracy.png



















Documents

Date Document
25 Mar 2013 IPS Results

User Acceptance Test - Registrar

Objectives

For our client to test out the completed application. The following functionalities were tested (which is the completed application except Indoor Positioning System):

  • Queue Management System Module
  • Alerts Module
  • Hearing List Module
  • Services Module
  • Admin Module

Details

  • Participants: 4 Supreme Court Registrars
  • Venue: Vicinity of the Court
  • Date: 10th - 12th April 2013
  • Duration: 3 days

Method

  • We have packaged our project into an .APK file and submitted it to our client to test on.
  • Together with the .APK file, we sent them a copy of the bug defect tracker for them to log and defects found.
  • Cases had to be filled up into the server on a daily basis to ensure testing went through without any glitch.
  • The employees conducted the tests throughout the 3 days.
  • This test is specifically catered to the Registrars of the Supreme Court.

Data collection

Collecting of results

  • Clients filled up the defect tracker and sent it to the team at the end of every testing day.
  • These bugs was then assigned to the developers to fix
  • Developers had to fix the bugs and report back to the testers and Project Manager

Mobile Phone Models

These are the phone models used by the various Registrars:

  • HTC One X+, Andriod 4.1.1
  • Samsung S3 GT-I9305, Andriod version 4.1.1
  • Samsung S3 GT-I9300, Andriod version 4.1.2
  • Samsung S2 GT-I9100, Andriod 2.3.6

Test Cases

INNOX UATRegistrar TABLE.png







Test Results

INNOX BUG UATRegistrar GRAPH.png





















Bug Distribution

INNOX UATRegistrar Distribution.png

















Detailed Test Cases

Date Document
13 Apr 2013 Functionality Test Case
13 Apr 2013 Functionality Test Results
13 Apr 2013 Defect Tracker

Usability Testing Schedule


Sprint Function Planned Start Date Actual Start Date Remarks
Test 1 Navigation Within Application 11th January 11th January Completed
Test 2 Alerts Usability 1st February 1st February Completed
Test 3 Menu Navigation & Logout 8th February 8th February Completed
Test 4 Queue Management System 22th February 22th February Completed
Test 5 User Interface Test 01st April 01st April Completed
Test 6 Indoor Positioning System 12th April 12th April Completed


Test 1

Objective

  • User-friendliness of the mobile application
  • Basic Navigation within the application. If buttons make sense

Details

  • Participants: 5 SMU Students
  • Venue: SMU, SIS ILab
  • Date: 11th January 2013 (Friday)
  • Duration: 1 hour

Methodology

We gave the users the mobile phone application to play around with and asked them to navigate around the application and at the end of it, to take aprt in a survey that we had prepared.

Data Collection

  • Observation of Students during the Test
  • Survey form as well as recommendation of which Set the students preferred.


Documentation

Description Document
Survey Questions Questions.docx
Survey Results Results.xlsx

Test 2

Objective

  • User-friendliness of the mobile application
  • Clarity of the content that is displayed in the alert notification
  • To minimum the complexity of the user experience when using the alert function

Details

  • Participants: 8 SMU Students
  • Venue: SMU, SIS ILab
  • Date: 1st February 2013 (Friday)
  • Duration: 1 hour

Methodology

We gave the users 2 different ways in which they could access the Alerts tabs using the 2 sets of paper prototype mock ups that were created. We gave them a set of instructions to follow and the freedom to ask any questions as we went along.

Data Collection

  • Observation of Students during the Test
  • Survey form as well as recommendation of which Set the students preferred.

Preference

INNOX USERTEST ALERTS.png



















Documentation

Description Document
Survey Questions Questions.docx
Survey Results Results.xlsx
Set A Page 1
Set A Page 2
Set A Page 3
Set A Page 4
Set B Page 1
Set B Page 2

Test 3

How We Conducted the User Test

Objectives

• To determine design inconsistencies and usability problem areas within the user interface and content areas. Potential sources of error may include:

     o	Navigation errors – failure to locate functions, excessive keystrokes to complete a function, failure to follow recommended screen flow.    
     o	Presentation errors – failure to locate and properly act upon desired information in screens, selection errors due to labeling ambiguities.
     o	Control usage problems – improper toolbar or entry field usage.

• Exercise the mobile application under controlled test conditions with representative users. Data will be used to access whether usability goals regarding an effective, efficient, and well-received user interface have been achieved.

• Establish baseline user performance and user-satisfaction levels of the user interface for future usability evaluations.

Details

  • Participants: 10 SMU Undergraduates
  • Venue: SMU, SIS Level 3 iLab
  • Date: 8th Feb 2013 (Friday)
  • Duration: 1 hour (4pm to 5pm)

Method

We conducted the user test using a high-fidelity paper prototype using the mock up designs of the application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure. We will be also taking down spontaneous comments made by the participants.
  • Upon completion of the paper prototype,we will ask the participants if they would like to see any changes to the current design of the paper prototype.

Features Tested

INNOX USERTEST3 Features.png









Results

Observations

INNOX USERTEST3 Obervations.png

















Comments

INNOX USERTEST3 Comments.png















Changes Made

1st Change

INNOX Usertestv3 Initial1.jpg INNOX To.jpg INNOX Usertestv3 Change1.jpg
Main Map Page New Main Map Page
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

Documents

Description Document
Usability Test Plan Test_Plan.docx
Test Results Results.xlsx
Note Taker's Guide Notetaker.doc
User Guide User_Guide.doc

Gallery

   

INNOX USERTESTv3 1.jpg

INNOX USERTESTv3 2.jpg

INNOX USERTESTv3 3.jpg

INNOX USERTESTv3 4.jpg

INNOX USERTESTv3 5.jpg

INNOX USERTESTv3 6.jpg

Test 4

How We Conducted the User Test

Objectives

• To determine design inconsistencies and usability problem areas within the user interface and content areas. Potential sources of error may include:

     o	Navigation errors – failure to locate functions, excessive keystrokes to complete a function, failure to follow recommended screen flow.    
     o	Presentation errors – failure to locate and properly act upon desired information in screens, selection errors due to labeling ambiguities.
     o	Control usage problems – improper toolbar or entry field usage.

• Exercise the mobile application under controlled test conditions with representative users. Data will be used to access whether usability goals regarding an effective, efficient, and well-received user interface have been achieved.

• Establish baseline user performance and user-satisfaction levels of the user interface for future usability evaluations.

Details

  • Participants: 10 SMU Undergraduates
  • Venue: SMU, SIS Level 3 iLab
  • Date: 22nd Feb 2013 (Friday)
  • Duration: 1 hour (1pm-2pm)

Method

We conducted the user test using a Samsung S3 phone for the mobile application. Thereafter, we gave the participants the freedom to explore the application by themselves.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure. We will be also taking down spontaneous comments made by the participants.
  • Upon completion of the user test, we will ask the participants if they would like to see any changes to the current design of the mobile application which they think is necessary to be implemented.

Features Tested

INNOX USERTEST4 Features.png











Results

Observations

INNOX USERTEST4 Obervations.png










Comments

INNOX USERTEST4 Comments.png
















Changes Made

Current Issue

  • Currently push alert notification when it is being sent to the phone, there is no sound or vibration to alert the user
  • User tend to miss the alert notification as there is no pop up

Changes to be implemented

  • Sound and vibration will be implemented for the alert notification

Documents

Description Document
Usability Test Plan Test_Plan.docx
Test Results Results.xlsx
Note Taker's Guide Notetaker.doc
User Guide User_Guide.doc

Gallery

   

INNOX USERTESTv4 11.jpg

INNOX USERTESTv4 2.jpg

INNOX USERTESTv4 3.jpg

INNOX USERTESTv4 15.jpg

INNOX USERTESTv4 5.jpg

INNOX USERTESTv4 6.jpg

Test 5

How We Conducted the User Test

Objectives

• To determine design inconsistencies and usability problem areas within the user interface and content areas. Potential sources of error may include:

     o	Navigation errors – failure to locate functions, excessive keystrokes to complete a function, failure to follow recommended screen flow.    
     o	Presentation errors – failure to locate and properly act upon desired information in screens, selection errors due to labeling ambiguities.
     o	Control usage problems – improper toolbar or entry field usage.

• Exercise the mobile application under controlled test conditions with representative users. Data will be used to access whether usability goals regarding an effective, efficient, and well-received user interface have been achieved.

• Establish baseline user performance and user-satisfaction levels of the user interface for future usability evaluations.

Details

  • Participants: 10 SMU Undergraduates
  • Venue: SMU, SIS Level 3 iLab
  • Date: 1 April 2013 (Monday)
  • Duration: 1 hour (1pm to 2pm)

Method

We conducted the user test using the following phones:

  • Samsung S3 GT-19300
  • Samsung S2 GT-19200
  • Samsung Note 2 N7105
  • Samsung Note N7000
  • LG Optimus 2x

Thereafter, we gave the participants the freedom to explore the application by themselves. A set of scenario are given to them so that concurrently there are 2 to 3 users accessing the mobile application at the same time.

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure. We will be also taking down spontaneous comments made by the participants.
  • Upon completion of the user test, we will ask the participants if they would like to see any changes to the current design of the mobile application which they think is necessary to be implemented.

Features Tested

INNOX USERTEST5 Features.png







Results

Observations

INNOX USERTEST5 Obervations.png








Comments

INNOX USERTEST5 Comments.png











Graphs

Innox Graph1.png Innox Graph2.png Innox Graph3.png

Changes Made

1st Change

INNOX Usertestv5 Initial1.png INNOX To.jpg INNOX Usertestv5 Change1.png
Hearings list Page New Hearings list Page
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

2nd Change

INNOX Usertestv5 Initial2.png INNOX To.jpg INNOX Usertestv5 Change2.png
View Queue Status Page New View Queue Status Page
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

3rd Change

INNOX Usertestv5 Initial3.png INNOX To.jpg INNOX Usertestv5 Change3.png
Queue Registration Page New Queue Registration Page
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

4th Change

INNOX Usertestv5 Initial4.png INNOX To.jpg INNOX Usertestv5 Change4.png
Login Page New Login Page
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

Documents

Description Document
Usability Test Plan Test_Plan.docx
Test Results Results.xlsx
Note Taker's Guide Notetaker.doc
User Guide User_Guide.doc

Gallery

   

INNOX USERTESTv5 1.jpg

INNOX USERTESTv5 2.jpg

INNOX USERTESTv5 3.jpg

INNOX USERTESTv5 4.jpg

INNOX USERTESTv5 5.jpg

INNOX USERTESTv5 6.jpg

Test 6

How We Conducted the User Test

Objectives

• To determine design inconsistencies and usability problem areas within the user interface and content areas. Potential sources of error may include:

     o	Navigation errors – failure to locate functions, excessive keystrokes to complete a function, failure to follow recommended screen flow.    
     o	Presentation errors – failure to locate and properly act upon desired information in screens, selection errors due to labeling ambiguities.
     o	Control usage problems – improper toolbar or entry field usage.

• Exercise the mobile application under controlled test conditions with representative users. Data will be used to access whether usability goals regarding an effective, efficient, and well-received user interface have been achieved.

• Establish baseline user performance and user-satisfaction levels of the user interface for future usability evaluations.

Details

  • Participants: 10 SMU Undergraduates
  • Venue: SMU, SIS Level 3 iLab
  • Date: 12th April 2013 (Friday)
  • Duration: 1 hour (2pm to 3pm)

Method

We conducted the user test using the following phones:

  • Samsung S3 GT-19300
  • Samsung S2 GT-19200
  • Samsung Note 2 N7105
  • Samsung Note N7000
  • LG Optimus 2x

Thereafter, we gave the participants the freedom to explore the application by themselves. A set of scenario are given to them so that user can use the mobile application for them to route to a specific chamber using the indoor system position map

Data collection

Collecting of qualitative metrics

  • We will observe how the participants use the application during the testing procedure. We will be also taking down spontaneous comments made by the participants.
  • Upon completion of the test,we will ask the participants if they would like to see any changes to the current design of the indoor position system map and its functions.

Features Tested

  • Indoor Position System - Provide direction for user to route to a specific destination within the Singapore Supreme Court

Results

  • User would like to see the changes to the current floor map as the current floor map is too plain and dull.

Changes Made

1st Change

INNOX Usertestv6 Initial1.png INNOX To.jpg INNOX Usertestv6 Change1.png
Floor Plan Page New Floor Plan Page
------------------------------------------------------------------------------- -------------------------------------------------------------------------------

Bug Metric

INNOX BUG METRIC2.png

















INNOX BUG Grpah1.png



























Here are the defects that were fixed during the Fixing stage from the System Integration Test.

INNOX BUG Grpah2.png



























Here are the defect that were fixed after the Client User Test


INNOX BUG Grpahv1.png



























Here are the defects that were fixed during the Fixing stage from the User Acceptance Test - Client.

INNOX BUG Grpahv2.png



























Here are the defect that were fixed after the User Acceptance Test - Registrar.