HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki:2018T1 FORD USER TESTING"

From IS480
Jump to navigation Jump to search
Line 161: Line 161:
 
<li><b>Date:</b>27 September 2018 - 1 October<br/></li>
 
<li><b>Date:</b>27 September 2018 - 1 October<br/></li>
 
<li><b>Iteration:</b>9<br/></li>
 
<li><b>Iteration:</b>9<br/></li>
 +
<li><b>Location: </b>PSA Help Desk Office<br/></li>
 +
<li><b>Participants: </b>6 PSA help desk staff<br/></li>
 +
<li><b>Testing system: </b>Windows<br/></li>
 +
<li><b>Testing browser: </b> Google Chrome <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Objectives===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test all the developed functionalities for both Chatbot and dashboard</li>
 +
<li>To test the accuracy rate of the Chatbot</li>
 +
<li>To test whether the functionalities and user experience meet the expectations of the clients</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Admin Portal
 +
|-
 +
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|style="text-align: center;"| Authentication
 +
|-
 +
 +
|style="text-align: center;"| Provide User Guide
 +
|style="text-align: center;"| Dashboard (Hits Analysis, Category Analysis, Top Question Analysis)
 +
|-
 +
 +
|style="text-align: center;"| Check Vessel/Container Status
 +
|style="text-align: center;"| FAQ Database Modification on UI
 +
|-
 +
 +
|style="text-align: center;"| Feedback
 +
|style="text-align: center;"| FAQ Database Filter by Category
 +
|-
 +
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Feedback Management
 +
|-
 +
|}
 +
</center>
 +
 +
=== Test Instruction Form ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Admin Portal Testing: </b> Instructions can be found [https://goo.gl/forms/3MHMgQkwyJeJ9PHb2 here] <br/></li>
 +
<li><b>Chatbot Testing:  </b> Instructions can be found [https://goo.gl/forms/ajww9jvKNVAtJrqw2 here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Key Comments Raised by the Help Desk Staff ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Chatbot
 +
|style="text-align: left;"| 1. User guide should be opened in a new tab instead of the original page, since the original user guide page contains the chat history <br> 2. Submission of the feedback should not refresh the chatbot page, which will clear the chat history
 +
3. The Chatbot did not provide the answer for some of the testers’ queries
 +
|-
 +
|}
 +
</center>
 +
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>The admin portal is fully functioning with 0 bug detected. Its performance meets the expectation of the clients.</li>
 +
<li>Minor user experience issues are identified for chatbot, and the accuracy rate can be further improved. These changes will be discussed and implemented during the following iterations.</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>UAT 3</font></div> ==
 +
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date:</b>1 November - 15 November 2018<br/></li>
 +
<li><b>Iteration:</b>12<br/></li>
 
<li><b>Location: </b>PSA Help Desk Office<br/></li>
 
<li><b>Location: </b>PSA Help Desk Office<br/></li>
 
<li><b>Participants: </b>6 PSA help desk staff<br/></li>
 
<li><b>Participants: </b>6 PSA help desk staff<br/></li>

Revision as of 23:46, 14 November 2018

FordLogo.png

Ford home logo.png   HOME

 

Ford about us.png   ABOUT US

 

Ford project overview.png   PROJECT OVERVIEW

 

Ford project management.png   PROJECT MANAGEMENT

 

Ford documentation.png   DOCUMENTATION

 



UAT 1

Testing Plan

  • Date: 6 Aug 2018
  • Iteration: 5
  • Location: PSA Help Desk Office
  • Participants: 6 PSA help desk staff
  • Testing system: Windows
  • Testing browser: Google Chrome

Objectives

  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot
  • To test whether the functionalities and user experience meet expectations of the clients

Scope

Chatbot Admin Portal
Portnet Related Authentication
Enquiries Dashboard - Hits Analysis
API Module Dashboard - Category Analysis
Account Management - Change Password
Account Management - Add New User
Data Management - Bootstrap

Test Instruction Form

  • Admin Portal Testing: Instructions can be found here
  • Chatbot Testing: Instructions can be found here

Key Comments Raised by the Help Desk Staff

Chatbot Chatbot logic need to be improved to handle some corner questions, such as specific error code.
Admin Portal 1. Dashboard should be more informative, eg dual axis for hit analysis graph

2. Time frame filtering function should be added, which allows users to filter by day, week, year

Overall Result

  • The admin portal is fully functioning with 0 bug detected.
  • There are minor logic errors found for chatbot, which will be fixed during next iteration.

UAT 2

Testing Plan

  • Date:27 September 2018 - 1 October
  • Iteration:9
  • Location: PSA Help Desk Office
  • Participants: 6 PSA help desk staff
  • Testing system: Windows
  • Testing browser: Google Chrome

Objectives

  • To test all the developed functionalities for both Chatbot and dashboard
  • To test the accuracy rate of the Chatbot
  • To test whether the functionalities and user experience meet the expectations of the clients

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Provide User Guide Dashboard (Hits Analysis, Category Analysis, Top Question Analysis)
Check Vessel/Container Status FAQ Database Modification on UI
Feedback FAQ Database Filter by Category
Feedback Management

Test Instruction Form

  • Admin Portal Testing: Instructions can be found here
  • Chatbot Testing: Instructions can be found here

Key Comments Raised by the Help Desk Staff

Chatbot 1. User guide should be opened in a new tab instead of the original page, since the original user guide page contains the chat history
2. Submission of the feedback should not refresh the chatbot page, which will clear the chat history

3. The Chatbot did not provide the answer for some of the testers’ queries

Overall Result

  • The admin portal is fully functioning with 0 bug detected. Its performance meets the expectation of the clients.
  • Minor user experience issues are identified for chatbot, and the accuracy rate can be further improved. These changes will be discussed and implemented during the following iterations.

UAT 3

Testing Plan

  • Date:1 November - 15 November 2018
  • Iteration:12
  • Location: PSA Help Desk Office
  • Participants: 6 PSA help desk staff
  • Testing system: Windows
  • Testing browser: Google Chrome

Objectives

  • To test all the developed functionalities for both Chatbot and dashboard
  • To test the accuracy rate of the Chatbot
  • To test whether the functionalities and user experience meet the expectations of the clients

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Provide User Guide Dashboard (Hits Analysis, Category Analysis, Top Question Analysis)
Check Vessel/Container Status FAQ Database Modification on UI
Feedback FAQ Database Filter by Category
Feedback Management

Test Instruction Form

  • Admin Portal Testing: Instructions can be found here
  • Chatbot Testing: Instructions can be found here

Key Comments Raised by the Help Desk Staff

Chatbot 1. User guide should be opened in a new tab instead of the original page, since the original user guide page contains the chat history
2. Submission of the feedback should not refresh the chatbot page, which will clear the chat history

3. The Chatbot did not provide the answer for some of the testers’ queries

Overall Result

  • The admin portal is fully functioning with 0 bug detected. Its performance meets the expectation of the clients.
  • Minor user experience issues are identified for chatbot, and the accuracy rate can be further improved. These changes will be discussed and implemented during the following iterations.