HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki:2018T1 FORD INTERNAL TESTING"

From IS480
Jump to navigation Jump to search
Line 64: Line 64:
 
<p>
 
<p>
 
<ul>
 
<ul>
<li><b>Date: </b>6 July<br/></li>
+
<li><b>Date: </b>6 July 2018<br/></li>
 
<li><b>Iteration: </b>3<br/></li>
 
<li><b>Iteration: </b>3<br/></li>
 
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
Line 124: Line 124:
 
<ul>
 
<ul>
 
<li>The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.</li>
 
<li>The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 2</font></div> ==
 +
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date: </b>26 July 2018<br/></li>
 +
<li><b>Iteration: </b>4<br/></li>
 +
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 +
 +
<li><b>Testing system: </b>Windows<br/></li>
 +
<li><b>Testing platform: </b>Google Chrome</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Objectives===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test all the developed functionalities for both chatbot and dashboard</li>
 +
<li>To test the accuracy rate of the chatbot</li>
 +
<li>To ensure the integration for chatbot and dashboard is successful</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal
 +
|-
 +
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|style="text-align: center;"| Authentication
 +
|-
 +
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Dashboard - Hits Analysis
 +
|-
 +
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Dashboard - Category Analysis
 +
|-
 +
|}
 +
</center>
 +
 +
=== Test cases & Test results ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
<ul>
 +
<li>Instructions can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit#gid=1910662255 here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: left;"| Chatbot
 +
| style="text-align: left;"| Accuracy rate: 77%
 +
| style="text-align: left;"| The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call.
 +
|-
 +
| style="text-align: left;"| Admin Portal
 +
| style="text-align: left;"| Dashboard is behaving as expected except minor error.
 +
| style="text-align: left;"| Slight changes of UI design may be proposed after a demonstration with the client.
 +
|-
 +
|}
 +
</center>
 +
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>With more testing question developed, the chatbot accuracy rate dropped. </li>
 +
<li>This is expected and the accuracy rate is still within a satisfactory range. </li>
 
</ul>
 
</ul>
 
</p>
 
</p>
 
</div>
 
</div>

Revision as of 00:44, 21 August 2018

FordLogo.png

Ford home logo.png   HOME

 

Ford about us.png   ABOUT US

 

Ford project overview.png   PROJECT OVERVIEW

 

Ford project management.png   PROJECT MANAGEMENT

 

Ford documentation.png   DOCUMENTATION

 



Internal Testing 1

Testing Plan

  • Date: 6 July 2018
  • Iteration: 3
  • Location: SMU SIS GSR 2.3
  • Participants: 6 PSA help desk staff
  • Testing system: Windows
  • Testing platform: Google Chrome

Objective

  • To test the accuracy rate of the chatbot

Scope

Chatbot
Portnet Related Enquiries

Test cases & Test results

  • Instructions can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Accuracy rate: 85% Develop more testing questions to expand the testing coverage for next round of testing

Overall Result

  • The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.

Internal Testing 2

Testing Plan

  • Date: 26 July 2018
  • Iteration: 4
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot
  • To ensure the integration for chatbot and dashboard is successful

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Dashboard - Hits Analysis
Dashboard - Category Analysis

Test cases & Test results

  • Instructions can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Accuracy rate: 77% The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call.
Admin Portal Dashboard is behaving as expected except minor error. Slight changes of UI design may be proposed after a demonstration with the client.

Overall Result

  • With more testing question developed, the chatbot accuracy rate dropped.
  • This is expected and the accuracy rate is still within a satisfactory range.