HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki:2018T1 FORD INTERNAL TESTING"

From IS480
Jump to navigation Jump to search
(update)
 
(19 intermediate revisions by one other user not shown)
Line 58: Line 58:
 
|}
 
|}
 
<br>
 
<br>
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>UAT 1</font></div> ==
+
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 1</font></div> ==
  
 
=== Testing Plan ===
 
=== Testing Plan ===
Line 64: Line 64:
 
<p>
 
<p>
 
<ul>
 
<ul>
<li><b>Date: </b>6 July<br/></li>
+
<li><b>Date: </b>6 July 2018<br/></li>
 
<li><b>Iteration: </b>3<br/></li>
 
<li><b>Iteration: </b>3<br/></li>
 
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
<li><b>Participants: </b>6 PSA help desk staff<br/></li>
+
<li><b>Testing system: </b>Windows<br/></li>
 +
<li><b>Testing platform: </b>Google Chrome</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Objective===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test the accuracy rate of the chatbot</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
|-
 +
 
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|-
 +
 
 +
|}
 +
</center>
 +
 
 +
=== Test cases & Test results ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
<ul>
 +
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit#gid=0 here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: left;"| Chatbot
 +
| style="text-align: left;"| Accuracy rate: 85%
 +
| style="text-align: left;"| Develop more testing questions to expand the testing coverage for next round of testing
 +
|-
 +
|}
 +
</center>
 +
 
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 2</font></div> ==
 +
 
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date: </b>26 July 2018<br/></li>
 +
<li><b>Iteration: </b>4<br/></li>
 +
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 +
 
 +
<li><b>Testing system: </b>Windows<br/></li>
 +
<li><b>Testing platform: </b>Google Chrome</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Objectives===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test all the developed functionalities for both chatbot and dashboard</li>
 +
<li>To test the accuracy rate of the chatbot</li>
 +
<li>To ensure the integration for chatbot and dashboard is successful</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal
 +
|-
 +
 
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|style="text-align: center;"| Authentication
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Dashboard - Hits Analysis
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Dashboard - Category Analysis
 +
|-
 +
|}
 +
</center>
 +
 
 +
=== Test cases & Test results ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
<ul>
 +
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit#gid=1910662255 here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: left;"| Chatbot
 +
| style="text-align: left;"| Accuracy rate: 77%
 +
| style="text-align: left;"| The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call.
 +
|-
 +
| style="text-align: left;"| Admin Portal
 +
| style="text-align: left;"| Dashboard is behaving as expected except minor error.
 +
| style="text-align: left;"| Slight changes of UI design may be proposed after a demonstration with the client.
 +
|-
 +
|}
 +
</center>
 +
 
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>With more testing question developed, the chatbot accuracy rate dropped. </li>
 +
<li>This is expected and the accuracy rate is still within a satisfactory range. </li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
 
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 3</font></div> ==
 +
 
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date: </b>4 Aug 2018<br/></li>
 +
<li><b>Iteration: </b>5<br/></li>
 +
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 +
 
 
<li><b>Testing system: </b>Windows<br/></li>
 
<li><b>Testing system: </b>Windows<br/></li>
 
<li><b>Testing platform: </b>Google Chrome</li>
 
<li><b>Testing platform: </b>Google Chrome</li>
Line 78: Line 234:
 
<p>
 
<p>
 
<ul>
 
<ul>
 +
<li>To test whether the deployment is successful</li>
 
<li>To test all the developed functionalities for both chatbot and dashboard</li>
 
<li>To test all the developed functionalities for both chatbot and dashboard</li>
 
<li>To test the accuracy rate of the chatbot</li>
 
<li>To test the accuracy rate of the chatbot</li>
<li>To test whether the functionalities and user experience meet expectations of the clients</li>
 
 
</ul>
 
</ul>
 
</p>
 
</p>
Line 90: Line 246:
 
|-
 
|-
 
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Admin Portal
+
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal
 
|-
 
|-
  
|style="text-align: center;"| Portnet Related  
+
|style="text-align: center;"| Portnet Related Enquiries
 
|style="text-align: center;"| Authentication
 
|style="text-align: center;"| Authentication
 
|-
 
|-
Line 119: Line 275:
 
</center>
 
</center>
  
=== Test Instruction Form ===
+
=== Test cases & Test results ===
 
<div style="font-family: Verdana;">
 
<div style="font-family: Verdana;">
 
<p>
 
<p>
 
<ul>
 
<ul>
<li><b>Admin Portal Testing: </b> Instructions can be found [https://goo.gl/forms/Ut98qh5GxBe9sFzM2 here] <br/></li>
+
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit#gid=1888806492 here] <br/></li>
<li><b>Chatbot Testing:  </b> Instructions can be found [https://goo.gl/forms/x8KkwrZWjECBugoG3 here] <br/></li>
 
 
</ul>
 
</ul>
 
</p>
 
</p>
 
</div>
 
</div>
  
=== Key Comments Raised by the Help Desk Staff ===
+
=== Key Findings ===
 
<center>
 
<center>
 
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Chatbot
+
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
|style="text-align: left;"| Chatbot logic need to be improved to handle some corner questions, such as specific error code.
+
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: center;"| Chatbot
 +
| style="text-align: left;"| Test accuracy rate: 77%
 +
Minor error for API discovered, which is not critical.
 +
| style="text-align: left;"| Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
 +
|-
 +
| style="text-align: center;"| Admin Portal
 +
| style="text-align: left;"| Dashboard is behaving as expected.
 +
Account management is behaving as expected except minor error.
 +
| style="text-align: left;"| Slight changes of UI design may be proposed after a demonstration with the client.
 +
|-
 +
|}
 +
</center>
 +
 
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>With more testing question developed, the chatbot accuracy rate dropped. </li>
 +
<li>This is expected and the accuracy rate is still within a satisfactory range. </li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 4</font></div> ==
 +
 
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date: </b>5 Sept 2018<br/></li>
 +
<li><b>Iteration: </b>7<br/></li>
 +
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 +
 
 +
<li><b>Testing system: </b>Windows<br/></li>
 +
<li><b>Testing platform: </b>Google Chrome</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Objectives===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test whether the deployment is successful</li>
 +
<li>To test all the developed functionalities for both Chatbot and dashboard</li>
 +
<li>To test the accuracy rate of the Chatbot</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal
 +
|-
 +
 
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|style="text-align: center;"| Authentication
 +
|-
 +
 
 +
|style="text-align: center;"| Check Container/Vessel
 +
|style="text-align: center;"| Dashboard - Hits Analysis
 +
|-
 +
 
 +
|style="text-align: center;"| Check for User Guide
 +
|style="text-align: center;"| Dashboard - Category Analysis
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Dashboard - Filter by Time Frame
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Account Management - Change Password
 +
 
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Account Management - Add New User
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - File Upload
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - FAQ Management
 +
|-
 +
 
 +
|}
 +
</center>
 +
 
 +
=== Test cases & Test results ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
<ul>
 +
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit?usp=sharing here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: center;"| Chatbot
 +
| style="text-align: left;"| Test accuracy rate: 70%
 +
| style="text-align: left;"|
 +
|-
 +
| style="text-align: center;"| Admin Portal
 +
| style="text-align: left;"| Some error messages are not precise with bad alignment.
 +
| style="text-align: left;"| Refine all the error messages.
 +
|}
 +
</center>
 +
 
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>All test cases are passed. </li>
 +
<li>The introduction of intent matching affects the accuracy of Chatbot.  </li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 5</font></div> ==
 +
 
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date: </b>1 Oct 2018<br/></li>
 +
<li><b>Iteration: </b>9<br/></li>
 +
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 +
 
 +
<li><b>Testing system: </b> Windows<br/></li>
 +
<li><b>Testing platform: </b> Google Chrome</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Objectives===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test whether the deployment is successful</li>
 +
<li>To test all the developed functionalities for both chatbot and dashboard</li>
 +
<li>To test the accuracy rate of the Chatbot</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal
 +
|-
 +
 
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|style="text-align: center;"| Authentication
 +
|-
 +
 
 +
|style="text-align: center;"| Check Container/Vessel Status
 +
|style="text-align: center;"| Dashboard - Hits Analysis
 +
|-
 +
 
 +
|style="text-align: center;"| Check for User Guide
 +
|style="text-align: center;"| Dashboard - Category Analysis
 +
|-
 +
 
 +
|style="text-align: center;"| Feedback
 +
|style="text-align: center;"| Dashboard - Top Question Analysis
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Account Management - Change Password
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Account Management - Add New User
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - File Upload
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - FAQ Management
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Feedback Management
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Announcement Management
 +
|-
 +
 
 +
|}
 +
</center>
 +
 
 +
=== Test cases & Test results ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
<ul>
 +
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit?usp=sharing here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: center;"| Chatbot
 +
| style="text-align: left;"| Test accuracy rate: 80%
 +
| style="text-align: left;"| Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
 +
|-
 +
| style="text-align: center;"| Admin Portal
 +
| style="text-align: left;"| 1. Poor user experience for feedback function is detected. Eg. When a user submits a feedback, no confirmation message is displayed, the feedback window simply closed. Minor bugs for feedback display on the admin side is found.
 +
2. Minor issue for timestamp display
 +
| style="text-align: left;"| 1. Success message should be displayed after the user submits a feedback.
 +
2. Round up timestamp display to full number.
 +
|-
 +
|}
 +
</center>
 +
 
 +
=== Overall Result ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li>Overall good performance with one test case failed. </li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
== <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 6</font></div> ==
 +
 
 +
=== Testing Plan ===
 +
<div style="font-family: Verdana; ">
 +
<p>
 +
<ul>
 +
<li><b>Date: </b>1 Nov 2018<br/></li>
 +
<li><b>Iteration: </b>12<br/></li>
 +
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li>
 +
 
 +
<li><b>Testing system: </b> Windows<br/></li>
 +
<li><b>Testing platform: </b> Google Chrome</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Objectives===
 +
<div style="font-family: Verdana;>
 +
<p>
 +
<ul>
 +
<li>To test whether the deployment is successful</li>
 +
<li>To test all the developed functionalities for both chatbot and dashboard</li>
 +
<li>To test the accuracy rate of the Chatbot</li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Scope ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
|-
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal
 +
|-
 +
 
 +
|style="text-align: center;"| Portnet Related Enquiries
 +
|style="text-align: center;"| Authentication
 +
|-
 +
 
 +
|style="text-align: center;"| Check Container/Vessel Status
 +
|style="text-align: center;"| Dashboard - Hits Analysis
 +
|-
 +
 
 +
|style="text-align: center;"| Check for User Guide
 +
|style="text-align: center;"| Dashboard - Category Analysis
 +
|-
 +
 
 +
|style="text-align: center;"| Feedback
 +
|style="text-align: center;"| Dashboard - Top Question Analysis
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Account Management - Change Password
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Account Management - Add New User
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - File Upload
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Data Management - FAQ Management
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Feedback Management
 +
|-
 +
 
 +
|style="text-align: center;"|
 +
|style="text-align: center;"| Announcement Management
 +
|-
 +
 
 +
|}
 +
</center>
 +
 
 +
=== Test cases & Test results ===
 +
<div style="font-family: Verdana;">
 +
<p>
 +
<ul>
 +
<li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit?usp=sharing here] <br/></li>
 +
</ul>
 +
</p>
 +
</div>
 +
 
 +
=== Key Findings ===
 +
<center>
 +
{|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center"
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour
 +
! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed
 +
|-
 +
| style="text-align: center;"| Chatbot
 +
| style="text-align: left;"| Test accuracy rate: 80%. When HTTPS is implemented, the application behaviour slightly varies.Unsafe script need to be loaded for the application to fully function.
 +
| style="text-align: left;"| Try to find free CA signed HTTPS certificate.
 
|-
 
|-
! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Admin Portal
+
| style="text-align: center;"| Admin Portal
|style="text-align: left;"| Dashboard should be more informative, eg dual axis for hit analysis graph
+
| style="text-align: left;"| Bug found during to question index issue
Time frame filtering function should be added, which allows users to filter by day, week, month
+
| style="text-align: left;"| Improve the database structurer.
 
|-
 
|-
 
|}
 
|}
Line 146: Line 646:
 
<p>
 
<p>
 
<ul>
 
<ul>
<li>The admin portal is fully functioning with 0 bug detected.</li>
+
<li>Overall good performance with one test case failed. </li>
<li>There are minor logic errors found for chatbot, which will be fixed during next iteration.</li>
 
 
</ul>
 
</ul>
 
</p>
 
</p>
 
</div>
 
</div>

Latest revision as of 15:48, 15 November 2018

FordLogo.png

Ford home logo.png   HOME

 

Ford about us.png   ABOUT US

 

Ford project overview.png   PROJECT OVERVIEW

 

Ford project management.png   PROJECT MANAGEMENT

 

Ford documentation.png   DOCUMENTATION

 



Internal Testing 1

Testing Plan

  • Date: 6 July 2018
  • Iteration: 3
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objective

  • To test the accuracy rate of the chatbot

Scope

Chatbot
Portnet Related Enquiries

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Accuracy rate: 85% Develop more testing questions to expand the testing coverage for next round of testing

Overall Result

  • The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.

Internal Testing 2

Testing Plan

  • Date: 26 July 2018
  • Iteration: 4
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot
  • To ensure the integration for chatbot and dashboard is successful

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Dashboard - Hits Analysis
Dashboard - Category Analysis

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Accuracy rate: 77% The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call.
Admin Portal Dashboard is behaving as expected except minor error. Slight changes of UI design may be proposed after a demonstration with the client.

Overall Result

  • With more testing question developed, the chatbot accuracy rate dropped.
  • This is expected and the accuracy rate is still within a satisfactory range.


Internal Testing 3

Testing Plan

  • Date: 4 Aug 2018
  • Iteration: 5
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Enquiries Dashboard - Hits Analysis
API Module Dashboard - Category Analysis
Account Management - Change Password
Account Management - Add New User
Data Management - Bootstrap

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 77%

Minor error for API discovered, which is not critical.

Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
Admin Portal Dashboard is behaving as expected.

Account management is behaving as expected except minor error.

Slight changes of UI design may be proposed after a demonstration with the client.

Overall Result

  • With more testing question developed, the chatbot accuracy rate dropped.
  • This is expected and the accuracy rate is still within a satisfactory range.

Internal Testing 4

Testing Plan

  • Date: 5 Sept 2018
  • Iteration: 7
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both Chatbot and dashboard
  • To test the accuracy rate of the Chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Check Container/Vessel Dashboard - Hits Analysis
Check for User Guide Dashboard - Category Analysis
Dashboard - Filter by Time Frame
Account Management - Change Password
Account Management - Add New User
Data Management - File Upload
Data Management - FAQ Management

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 70%
Admin Portal Some error messages are not precise with bad alignment. Refine all the error messages.

Overall Result

  • All test cases are passed.
  • The introduction of intent matching affects the accuracy of Chatbot.

Internal Testing 5

Testing Plan

  • Date: 1 Oct 2018
  • Iteration: 9
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the Chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Check Container/Vessel Status Dashboard - Hits Analysis
Check for User Guide Dashboard - Category Analysis
Feedback Dashboard - Top Question Analysis
Account Management - Change Password
Account Management - Add New User
Data Management - File Upload
Data Management - FAQ Management
Feedback Management
Announcement Management

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 80% Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call.
Admin Portal 1. Poor user experience for feedback function is detected. Eg. When a user submits a feedback, no confirmation message is displayed, the feedback window simply closed. Minor bugs for feedback display on the admin side is found.

2. Minor issue for timestamp display

1. Success message should be displayed after the user submits a feedback.

2. Round up timestamp display to full number.

Overall Result

  • Overall good performance with one test case failed.

Internal Testing 6

Testing Plan

  • Date: 1 Nov 2018
  • Iteration: 12
  • Location: SMU SIS GSR 2.3
  • Testing system: Windows
  • Testing platform: Google Chrome

Objectives

  • To test whether the deployment is successful
  • To test all the developed functionalities for both chatbot and dashboard
  • To test the accuracy rate of the Chatbot

Scope

Chatbot Admin Portal
Portnet Related Enquiries Authentication
Check Container/Vessel Status Dashboard - Hits Analysis
Check for User Guide Dashboard - Category Analysis
Feedback Dashboard - Top Question Analysis
Account Management - Change Password
Account Management - Add New User
Data Management - File Upload
Data Management - FAQ Management
Feedback Management
Announcement Management

Test cases & Test results

  • Test cases & Test results can be found here

Key Findings

Functionality Application Behaviour Changes proposed
Chatbot Test accuracy rate: 80%. When HTTPS is implemented, the application behaviour slightly varies.Unsafe script need to be loaded for the application to fully function. Try to find free CA signed HTTPS certificate.
Admin Portal Bug found during to question index issue Improve the database structurer.

Overall Result

  • Overall good performance with one test case failed.