HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki:2017T1 Ducky King User Testing 2"

From IS480
Jump to navigation Jump to search
 
(20 intermediate revisions by 3 users not shown)
Line 44: Line 44:
 
<!-- Current Page -->
 
<!-- Current Page -->
 
| style="vertical-align:top;width:14.28%;" | <div style="padding: 3px;line-height: wrap_content;  border-bottom:5px solid #FF7505; font-family:helvetica"> [[IS480_Team_wiki:2017T1 Ducky King User Testing| <font color =#535353><b>User Testing</b></font>]]
 
| style="vertical-align:top;width:14.28%;" | <div style="padding: 3px;line-height: wrap_content;  border-bottom:5px solid #FF7505; font-family:helvetica"> [[IS480_Team_wiki:2017T1 Ducky King User Testing| <font color =#535353><b>User Testing</b></font>]]
 
| style="vertical-align:top;width:11.28%;" | <div style="padding: 3px; font-weight: bold; text-align:center; line-height: wrap_content; border-bottom:1px dashed #FF7505; font-family:helvetica"> [[IS480_Team_wiki:2017T1 Ducky King Prototype | <font color =#FF7505><b>Prototype</b></font>]]
 
  
 
| style="vertical-align:top;width:15.28%;" | <div style="padding: 3px; font-weight: bold; text-align:center; line-height: wrap_content; border-bottom:1px dashed #FF7505; font-family:helvetica"> [[IS480_Team_wiki:2017T1 Ducky King Project Deliverables | <font color =#FF7505><b>Project Deliverables</b></font>]]
 
| style="vertical-align:top;width:15.28%;" | <div style="padding: 3px; font-weight: bold; text-align:center; line-height: wrap_content; border-bottom:1px dashed #FF7505; font-family:helvetica"> [[IS480_Team_wiki:2017T1 Ducky King Project Deliverables | <font color =#FF7505><b>Project Deliverables</b></font>]]
Line 228: Line 226:
 
</p>
 
</p>
 
</div>
 
</div>
 +
 +
===User Interface used in UAT 2===
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
 +
File:DK_flowadmin_uat2_login.PNG.jpg | FlowAdmin Login Page
 +
File:DK_flowadmin_uat2_dashboard.PNG.jpg | FlowAdmin Dashboard
 +
File:DK_flowadmin_uat2_monitor.PNG.jpg | FlowAdmin Organization Monitor
 +
File:DK_flowadmin_uat2_auctions.PNG.jpg | FlowAdmin View All Auctions Page
 +
File:DK_flowadmin_uat2_auctions-details.PNG.jpg | FlowAdmin View Auction Details Page
 +
 +
</gallery><br>
  
 
===Environment===
 
===Environment===
Line 242: Line 251:
 
</p>
 
</p>
 
</div>
 
</div>
 +
=== UAT 2 Results ===
 
<center>
 
<center>
 
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
Line 265: Line 275:
 
|style="text-align: left;"| Goal Reached. All users were able to delete a specific bid
 
|style="text-align: left;"| Goal Reached. All users were able to delete a specific bid
 
|}
 
|}
 +
</center><br><br>
 +
 +
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="100pt" | Functions
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Users’ Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes to be made
 +
|-
 +
|style="text-align: center;"| User Interface
 +
|style="text-align: left;"| Participants were confused when bids were not listed in chronological order. This is due to the user interface sorting the bids from the earliest to the latest time.
 +
|style="text-align: left;"| The user interface would now instead sort the bids in reverse chronological order, from the latest bid to the earliest.
 +
|-
 +
|style="text-align: center;"| User Interface
 +
|style="text-align: left;"| The loading screen for the auction details shows an error status and an inaccurate auction expiry. This momentarily confuses the participants before the screen finishes loading. 
 +
|style="text-align: left;"| The user interface should instead have no default values to avoid confusing the users.
 +
|-
 +
|style="text-align: center;"| User Interface
 +
|style="text-align: left;"| Even though there was a loading screen when deleting a bid, as the operation took a slightly longer time, the participants were unsure if the web application was still running.
 +
|style="text-align: left;"| The longer loading time is due to the nature of the blockchain. We will be conducting scalability study after the midterms to research on optimization methods.
 +
|}
 +
 +
</center>
 +
 +
=== Changes from UAT 1 ===
 +
<center>
 +
We added the listing of all bids as well as included their timestamps to make it easier to identify spam bids based on feedback from UAT 1.
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
File:DK_flowadmin_uat2_bid_timestamp.PNG | New FlowAdmin Bids Page
 +
</gallery><br>
 +
 +
We added the direction of the auctions to better distinguish the auction types as well as updated the column variables in the Auctions page to match the FlowLabs' front-end built by FlowLabs for consistency purposes.
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
File:DK Auction View HF.PNG | Before: FlowAdmin Auctions Page
 +
File:DK flowadmin uat2 auctions.PNG.jpg | Now: FlowAdmin Auctions Page
 +
</gallery><br>
 
</center>
 
</center>
  
 +
<!--[[File:DuckyKing UAT2 postquestionnaire.png|Post-Test Questionnaire Average Participant Feedback|600px]]
 +
[[File:DuckyKing Navigation Errors.png|300px]]-->
 
<!--/ Current Page -->
 
<!--/ Current Page -->
 +
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica"><font color= #FFFFFF>UAT 3</font></div> ==
 +
=== Test Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<b>Venue:</b> Flow Labs Office @ 165 Telok Ayer Street<br/>
 +
<b>Date:</b> 17 November 2017<br/>
 +
<b>Time:</b> 4.30PM<br/>
 +
<b>Duration:</b> ~ 20 minutes per user<br/>
 +
<b>Number of Participants for FlowAdmin:</b> TBC<br/>
 +
<b>User Test:</b> Instructions can be found [https://docs.google.com/document/d/11yC61dLuhneOx_jnXIANRBfa1sgmdNE0tHOkaj97EVc/edit#heading=h.m7sqzh4rdvdx here] <br/>
 +
</p>
 +
</div>
 +
 +
=== Objectives===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>To evaluate if the changes made to the interface from UAT#2 has increased the usability of FlowAdmin</li>
 +
<li>To detect usability issues based on target users’ behaviour</li>
 +
<li>To ensure that the system is what the user requires</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Scope of UAT 3===
 +
<div style="font-family: Verdana">
 +
<p>
 +
<ul>
 +
<li>Remove Peer</li>
 +
<li>Delete Bids</li>
 +
<li>Delete Auctions</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
===User Interface used in UAT 3===
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
 +
File:DK_flowadmin_login.PNG | FlowAdmin Login Page
 +
File:DK_flowadmin_dashboard.PNG | FlowAdmin Dashboard
 +
File:DK_flowadmin_monitor.PNG | FlowAdmin Organisation Monitor
 +
File:DK_flowadmin_auctions.PNG | FlowAdmin View All Auction Page
 +
File:DK_flowadmin_auctions-details.PNG | View Auction Details Page
 +
File:DK_flowadmin_bids.PNG | FlowAdmin View All Bids Page
 +
File:DK_flowadmin_peers.PNG | FlowAdmin View All Peers Page
 +
File:DK_flowadmin_notifications.PNG | FlowAdmin View All Notifications Page
 +
 +
</gallery><br>
 +
 +
===Environment===
 +
<p>
 +
As we want to conduct the study in the most realistic environment as a member of FlowLabs, we will be conducting our study at the Flowlabs office at 165 Telok Ayer St. We will be briefing the participants on what our application will be used for before they proceed with the tasks.
 +
 +
The participants will test the application using a laptop running Windows 8 and on the Google Chrome browser.
 +
</p>
 +
 +
=== Procedures ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
After the pre-screening of participants has been done, participants who pass the pre-test questionnaire will be given a consent form to sign prior to the testing. Once signed, they will be given the test instructions, and they are to think aloud as they follow through the instructions given for the user testing. Team Ducky King will be there to record down the participants’ behaviours. Participants will complete a post-test questionnaire at the end of their user testing. Team Ducky King will conclude the user testing by asking the participants questions with regards to their behaviours and thought process as they navigate through FlowAdmin platform.
 +
</p>
 +
</div>
 +
 +
=== UAT 3 Results ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="100pt" | S/N
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Goals
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Reached?
 +
|-
 +
|style="text-align: center;"| 1
 +
|style="text-align: left;"| User should be able to complete task #1 without guidance from a test facilitator
 +
|style="text-align: left;"| Goal Reached. All users were able to complete the task without guidance
 +
|-
 +
|style="text-align: center;"| 2
 +
|style="text-align: left;"|User should be able to easily remove a specific peer
 +
|style="text-align: left;"| Goal Reached. All users were able to remove a specific peer
 +
|-
 +
|style="text-align: center;"| 3
 +
|style="text-align: left;"| User should find it easy to delete a specific auction
 +
|style="text-align: left;"| Goal Reached. All users were able to delete a specific auction
 +
|-
 +
|style="text-align: center;"| 4
 +
|style="text-align: left;"| User should find it easy to delete a specific bid
 +
|style="text-align: left;"| Goal Reached. All users were able to delete a specific bid
 +
|}
 +
</center><br><br>
 +
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="100pt" | Functions
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Users’ Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes to be made
 +
|-
 +
|style="text-align: center;"| User Interface
 +
|style="text-align: left;"| The Peer Node IDs did not provide any meaningful information in identifying the peer/ node and was confusing.
 +
|style="text-align: left;"| The Peer Node IDs should be updated to be meaningful and useful in identifying the peer/ nodes.
 +
|-
 +
|style="text-align: center;"| User Interface
 +
|style="text-align: left;"| The pop-up notifications regarding the Peers did not clear away and cluttered the screen.
 +
|style="text-align: left;"| The pop-up notifications should only exist for a time period before fading away automatically.
 +
|}
 +
 +
</center>
 +
 +
=== Changes from UAT 2 ===
 +
<center>
 +
A new ‘Peers’ page was introduced to provide an overview of all the nodes connected to the FlowLabs blockchain.
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
File:DK_flowadmin_peers.PNG | FlowAdmin View All Peers Page
 +
</gallery><br>
 +
 +
A new ‘Notifications’ page was introduced to provide a listing of all notifications pertinent to the Flow Admin
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
File:DK_flowadmin_notifications.PNG | FlowAdmin View All Notifications Page
 +
</gallery><br>
 +
 +
The sorting for Bids has been changed to be sorted in reverse chronological order so that the newest bids will appear first
 +
<gallery mode=packed-overlay perrow=3 heights=180px>
 +
File:DK_flowadmin_uat2_bid_timestamp.PNG | Before: FlowAdmin Bids Page
 +
File:DK_flowadmin_bids.PNG | Now: FlowAdmin Bids Page
 +
</gallery><br>
 +
</center>

Latest revision as of 03:07, 21 November 2017

DuckyKingLogo.png


Duckyking home logo.png   HOME

 

Duckyking about us.png   ABOUT US

 

Duckyking project overview.png   PROJECT OVERVIEW

 

Duckyking project management.png   PROJECT MANAGEMENT

 

Ducky king documentation.png   DOCUMENTATION

 



FlowAdmin UAT Header.png



UAT 1

Test Plan

Venue: Flow Labs Office @ 165 Telok Ayer Street
Date: 15 August 2017
Time: 5.30PM
Duration: ~ 15 minutes per user
Number of Participants for FlowAdmin: 4
User Test: Instructions can be found here

Objectives

  • To gather feedback regarding user interface and the developed functions
  • To detect usability issues based on users’ behaviour
  • To improve web application interface based on the feedback

Scope of UAT 1

  • Delete Bids
  • Delete Auctions

High Fidelity Prototypes


Low Fidelity Prototypes


Procedures

Participants are given a consent form to sign prior to the testing. Once signed, they will be given the test instructions, and they are to think aloud as they follow through the instructions given for the user testing. Team Ducky King will be there to record down the participants’ behaviours. Participants will complete a post-test questionnaire at the end of their user testing. Team Ducky King will conclude the user testing by asking the participants questions with regards to their behaviours and thought process as they navigate through FlowAdmin platform.

S/N Goals Reached?
1 User should be able to complete the tasks without guidance from a test facilitator Goal reached. Participants were able to complete the tasks without guidance from a test facilitator
2 User should find it easy to remove a specific auction Goal reached. Participants were able to complete the tasks.
3 User should find it easy to remove a specific bid Goal reached. Participants were able to complete the tasks.

Key Findings

Functions Users’ Behaviour Changes to be made
User Interface Participants tend to click on the ‘delete auction’ and ‘delete bid’ buttons multiple times and do not know that the request is being processed. Introduce a loading spinner into the platform when the button is clicked so the user knows that the request has been sent.
User Interface Participants had a slight pause before being able to view the list of auctions. Put the navigation to the list of actions in a more visible position.
User Interface Participants had slight difficulty in finding the bids to delete from the respective auctions. Have a bids tab in the menu bar and a listing of all bids.
Functions Users’ Comment Changes to be made
Function Refinement Sponsor requested for a way to identify spam bids more easily using timestamps
  • Introduce the time stamps for easy identification of spam bids on the FlowAdmin platform
  • List all bids on 1 page

Overall Results/Comments

Based on the results and feedback from the UAT,

  • User interface can be improved to be more intuitive to identify dubious auctions and bids
  • Client was impressed by the FlowAdmin interface as they only required a command line interface

UAT 2

Test Plan

Venue: Flow Labs Office @ 165 Telok Ayer Street
Date: 9 October 2017
Time: 5.30PM
Duration: ~ 20 minutes per user
Number of Participants for FlowAdmin: 4
User Test: Instructions can be found here
Type of test: Laboratory Test

Objectives

  • To gather feedback regarding user interface and the developed functions
  • To detect usability issues based on target users’ behaviour
  • To improve web application interface based on the feedback

Scope of UAT 2

  • View FlowAdmin dashboard statistics
  • Delete Bids
  • Delete Auctions

User Interface used in UAT 2


Environment

As we want to conduct the study in the most realistic environment as a member of FlowLabs, we will be conducting our study at the Flowlabs office at 165 Telok Ayer St. We will be briefing the participants on what our application will be used for before they proceed with the tasks. The participants will test the application using a laptop running Windows 8 and on the Google Chrome browser.

Procedures

After the pre-screening of participants has been done, participants who pass the pre-test questionnaire will be given a consent form to sign prior to the testing. Once signed, they will be given the test instructions, and they are to think aloud as they follow through the instructions given for the user testing. Team Ducky King will be there to record down the participants’ behaviours. Participants will complete a post-test questionnaire at the end of their user testing. Team Ducky King will conclude the user testing by asking the participants questions with regards to their behaviours and thought process as they navigate through FlowAdmin platform.

UAT 2 Results

S/N Goals Reached?
1 User should be able to complete task #1 without guidance from a test facilitator Goal Reached. All users were able to complete the task without guidance
2 User should be able to easily find statistics on the auctions and bids Goal Reached. All users were able to find the statistics
3 User should find it easy to delete a specific auction Goal Reached. All users were able to delete a specific auction
4 User should find it easy to delete a specific bid Goal Reached. All users were able to delete a specific bid




Key Findings

Functions Users’ Behaviour Changes to be made
User Interface Participants were confused when bids were not listed in chronological order. This is due to the user interface sorting the bids from the earliest to the latest time. The user interface would now instead sort the bids in reverse chronological order, from the latest bid to the earliest.
User Interface The loading screen for the auction details shows an error status and an inaccurate auction expiry. This momentarily confuses the participants before the screen finishes loading. The user interface should instead have no default values to avoid confusing the users.
User Interface Even though there was a loading screen when deleting a bid, as the operation took a slightly longer time, the participants were unsure if the web application was still running. The longer loading time is due to the nature of the blockchain. We will be conducting scalability study after the midterms to research on optimization methods.

Changes from UAT 1

We added the listing of all bids as well as included their timestamps to make it easier to identify spam bids based on feedback from UAT 1.


We added the direction of the auctions to better distinguish the auction types as well as updated the column variables in the Auctions page to match the FlowLabs' front-end built by FlowLabs for consistency purposes.



UAT 3

Test Plan

Venue: Flow Labs Office @ 165 Telok Ayer Street
Date: 17 November 2017
Time: 4.30PM
Duration: ~ 20 minutes per user
Number of Participants for FlowAdmin: TBC
User Test: Instructions can be found here

Objectives

  • To evaluate if the changes made to the interface from UAT#2 has increased the usability of FlowAdmin
  • To detect usability issues based on target users’ behaviour
  • To ensure that the system is what the user requires

Scope of UAT 3

  • Remove Peer
  • Delete Bids
  • Delete Auctions

User Interface used in UAT 3


Environment

As we want to conduct the study in the most realistic environment as a member of FlowLabs, we will be conducting our study at the Flowlabs office at 165 Telok Ayer St. We will be briefing the participants on what our application will be used for before they proceed with the tasks. The participants will test the application using a laptop running Windows 8 and on the Google Chrome browser.

Procedures

After the pre-screening of participants has been done, participants who pass the pre-test questionnaire will be given a consent form to sign prior to the testing. Once signed, they will be given the test instructions, and they are to think aloud as they follow through the instructions given for the user testing. Team Ducky King will be there to record down the participants’ behaviours. Participants will complete a post-test questionnaire at the end of their user testing. Team Ducky King will conclude the user testing by asking the participants questions with regards to their behaviours and thought process as they navigate through FlowAdmin platform.

UAT 3 Results

S/N Goals Reached?
1 User should be able to complete task #1 without guidance from a test facilitator Goal Reached. All users were able to complete the task without guidance
2 User should be able to easily remove a specific peer Goal Reached. All users were able to remove a specific peer
3 User should find it easy to delete a specific auction Goal Reached. All users were able to delete a specific auction
4 User should find it easy to delete a specific bid Goal Reached. All users were able to delete a specific bid



Key Findings

Functions Users’ Behaviour Changes to be made
User Interface The Peer Node IDs did not provide any meaningful information in identifying the peer/ node and was confusing. The Peer Node IDs should be updated to be meaningful and useful in identifying the peer/ nodes.
User Interface The pop-up notifications regarding the Peers did not clear away and cluttered the screen. The pop-up notifications should only exist for a time period before fading away automatically.

Changes from UAT 2

A new ‘Peers’ page was introduced to provide an overview of all the nodes connected to the FlowLabs blockchain.


A new ‘Notifications’ page was introduced to provide a listing of all notifications pertinent to the Flow Admin


The sorting for Bids has been changed to be sorted in reverse chronological order so that the newest bids will appear first