HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki:2018T1 FORD INTERNAL TESTING"

From IS480
Jump to navigation Jump to search
Line 314: Line 314:
 
</div>
 
</div>
  
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 3</font></div> ==
+
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 4</font></div> ==
  
 
=== Testing Plan ===
 
=== Testing Plan ===
Line 320: Line 320:
 
<p>
 
<p>
 
<ul>
 
<ul>
<li><b>Date: </b>4 Aug 2018<br/></li>
+
<li><b>Date: </b>5 Sept 2018<br/></li>
<li><b>Iteration: </b>5<br/></li>
+
<li><b>Iteration: </b>7<br/></li>
 
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
  
Line 335: Line 335:
 
<ul>
 
<ul>
 
<li>To test whether the deployment is successful</li>
 
<li>To test whether the deployment is successful</li>
<li>To test all the developed functionalities for both chatbot and dashboard</li>
+
<li>To test all the developed functionalities for both Chatbot and dashboard</li>
<li>To test the accuracy rate of the chatbot</li>
+
<li>To test the accuracy rate of the Chatbot</li>
 
</ul>
 
</ul>
 
</p>
 
</p>
Line 353: Line 353:
 
|-
 
|-
  
|style="text-align: center;"| Enquiries
+
|style="text-align: center;"| Check Container/Vessel
 
|style="text-align: center;"| Dashboard - Hits Analysis
 
|style="text-align: center;"| Dashboard - Hits Analysis
 
|-
 
|-
  
|style="text-align: center;"| API Module
+
|style="text-align: center;"| Check for User Guide
 
|style="text-align: center;"| Dashboard - Category Analysis
 
|style="text-align: center;"| Dashboard - Category Analysis
 +
|-
 +
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Dashboard - Filter by Time Frame
 
|-
 
|-
  
 
|style="text-align: center;"|  
 
|style="text-align: center;"|  
 
|style="text-align: center;"| Account Management - Change Password
 
|style="text-align: center;"| Account Management - Change Password
 +
 
|-
 
|-
  
Line 370: Line 375:
  
 
|style="text-align: center;"|  
 
|style="text-align: center;"|  
|style="text-align: center;"| Data Management - Bootstrap
+
|style="text-align: center;"| Data Management - File Upload
 
|-
 
|-
 +
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - FAQ Management
 +
|-
 +
 
|}
 
|}
 
</center>
 
</center>
Line 379: Line 389:
 
<p>
 
<p>
 
<ul>
 
<ul>
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit#gid=1888806492 here] <br/></li>
+
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit?usp=sharing here] <br/></li>
 
</ul>
 
</ul>
 
</p>
 
</p>
Line 392: Line 402:
 
|-
 
|-
 
| style="text-align: center;"| Chatbot
 
| style="text-align: center;"| Chatbot
| style="text-align: left;"| Test accuracy rate: 77%
+
| style="text-align: left;"| Test accuracy rate: 70%
Minor error for API discovered, which is not critical.
+
| style="text-align: left;"|  
| style="text-align: left;"| Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
 
 
|-
 
|-
 
| style="text-align: center;"| Admin Portal
 
| style="text-align: center;"| Admin Portal
| style="text-align: left;"| Dashboard is behaving as expected.
+
| style="text-align: left;"| Some error messages are not precise with bad alignment.
Account management is behaving as expected except minor error.
+
| style="text-align: left;"| Refine all the error messages.
| style="text-align: left;"| Slight changes of UI design may be proposed after a demonstration with the client.
 
|-
 
 
|}
 
|}
 
</center>
 
</center>
Line 408: Line 415:
 
<p>
 
<p>
 
<ul>
 
<ul>
<li>With more testing question developed, the chatbot accuracy rate dropped. </li>
+
<li>All test cases are passed. </li>
<li>This is expected and the accuracy rate is still within a satisfactory range. </li>
+
<li>The introduction of intent matching affects the accuracy of Chatbot. </li>
 
</ul>
 
</ul>
 
</p>
 
</p>

Revision as of 21:22, 8 October 2018

FordLogo.png

Ford home logo.png   HOME

 

Ford about us.png   ABOUT US

 

Ford project overview.png   PROJECT OVERVIEW

 

Ford project management.png   PROJECT MANAGEMENT

 

Ford documentation.png   DOCUMENTATION

 



Internal Testing 1

Testing Plan

  • Date: 6 July 2018
  • Iteration: 3
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objective

  • To test the accuracy rate of the chatbot

Scope

Chatbot
Portnet Related Enquiries

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Accuracy rate: 85% Develop more testing questions to expand the testing coverage for next round of testing

Overall Result

  • The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.

Internal Testing 2

Testing Plan

  • Date: 26 July 2018
  • Iteration: 4
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot
  • To ensure the integration for chatbot and dashboard is successful

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Dashboard - Hits Analysis
Dashboard - Category Analysis

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Accuracy rate: 77% The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call.
Admin Portal Dashboard is behaving as expected except minor error. Slight changes of UI design may be proposed after a demonstration with the client.

Overall Result

  • With more testing question developed, the chatbot accuracy rate dropped.
  • This is expected and the accuracy rate is still within a satisfactory range.


Internal Testing 3

Testing Plan

  • Date: 4 Aug 2018
  • Iteration: 5
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Enquiries Dashboard - Hits Analysis
API Module Dashboard - Category Analysis
Account Management - Change Password
Account Management - Add New User
Data Management - Bootstrap

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 77%

Minor error for API discovered, which is not critical.

Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
Admin Portal Dashboard is behaving as expected.

Account management is behaving as expected except minor error.

Slight changes of UI design may be proposed after a demonstration with the client.

Overall Result

  • With more testing question developed, the chatbot accuracy rate dropped.
  • This is expected and the accuracy rate is still within a satisfactory range.

Internal Testing 4

Testing Plan

  • Date: 5 Sept 2018
  • Iteration: 7
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both Chatbot and dashboard
  • To test the accuracy rate of the Chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Check Container/Vessel Dashboard - Hits Analysis
Check for User Guide Dashboard - Category Analysis
Dashboard - Filter by Time Frame
Account Management - Change Password
Account Management - Add New User
Data Management - File Upload
Data Management - FAQ Management

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 70%
Admin Portal Some error messages are not precise with bad alignment. Refine all the error messages.

Overall Result

  • All test cases are passed.
  • The introduction of intent matching affects the accuracy of Chatbot.

Internal Testing 3

Testing Plan

  • Date: 4 Aug 2018
  • Iteration: 5
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Enquiries Dashboard - Hits Analysis
API Module Dashboard - Category Analysis
Account Management - Change Password
Account Management - Add New User
Data Management - Bootstrap

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 77%

Minor error for API discovered, which is not critical.

Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
Admin Portal Dashboard is behaving as expected.

Account management is behaving as expected except minor error.

Slight changes of UI design may be proposed after a demonstration with the client.

Overall Result

  • With more testing question developed, the chatbot accuracy rate dropped.
  • This is expected and the accuracy rate is still within a satisfactory range.