Difference between revisions of "IS480 Team wiki:2018T1 FORD INTERNAL TESTING"
Jump to navigation
Jump to search
Line 314: | Line 314: | ||
</div> | </div> | ||
− | == <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing | + | == <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 4</font></div> == |
=== Testing Plan === | === Testing Plan === | ||
Line 320: | Line 320: | ||
<p> | <p> | ||
<ul> | <ul> | ||
− | <li><b>Date: </b> | + | <li><b>Date: </b>5 Sept 2018<br/></li> |
− | <li><b>Iteration: </b> | + | <li><b>Iteration: </b>7<br/></li> |
<li><b>Location: </b>SMU SIS GSR 2.3<br/></li> | <li><b>Location: </b>SMU SIS GSR 2.3<br/></li> | ||
Line 335: | Line 335: | ||
<ul> | <ul> | ||
<li>To test whether the deployment is successful</li> | <li>To test whether the deployment is successful</li> | ||
− | <li>To test all the developed functionalities for both | + | <li>To test all the developed functionalities for both Chatbot and dashboard</li> |
− | <li>To test the accuracy rate of the | + | <li>To test the accuracy rate of the Chatbot</li> |
</ul> | </ul> | ||
</p> | </p> | ||
Line 353: | Line 353: | ||
|- | |- | ||
− | |style="text-align: center;"| | + | |style="text-align: center;"| Check Container/Vessel |
|style="text-align: center;"| Dashboard - Hits Analysis | |style="text-align: center;"| Dashboard - Hits Analysis | ||
|- | |- | ||
− | |style="text-align: center;"| | + | |style="text-align: center;"| Check for User Guide |
|style="text-align: center;"| Dashboard - Category Analysis | |style="text-align: center;"| Dashboard - Category Analysis | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Dashboard - Filter by Time Frame | ||
|- | |- | ||
|style="text-align: center;"| | |style="text-align: center;"| | ||
|style="text-align: center;"| Account Management - Change Password | |style="text-align: center;"| Account Management - Change Password | ||
+ | |||
|- | |- | ||
Line 370: | Line 375: | ||
|style="text-align: center;"| | |style="text-align: center;"| | ||
− | |style="text-align: center;"| Data Management - | + | |style="text-align: center;"| Data Management - File Upload |
|- | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Data Management - FAQ Management | ||
+ | |- | ||
+ | |||
|} | |} | ||
</center> | </center> | ||
Line 379: | Line 389: | ||
<p> | <p> | ||
<ul> | <ul> | ||
− | <li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit | + | <li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit?usp=sharing here] <br/></li> |
</ul> | </ul> | ||
</p> | </p> | ||
Line 392: | Line 402: | ||
|- | |- | ||
| style="text-align: center;"| Chatbot | | style="text-align: center;"| Chatbot | ||
− | | style="text-align: left;"| Test accuracy rate: | + | | style="text-align: left;"| Test accuracy rate: 70% |
− | + | | style="text-align: left;"| | |
− | | style="text-align: left;"| | ||
|- | |- | ||
| style="text-align: center;"| Admin Portal | | style="text-align: center;"| Admin Portal | ||
− | | style="text-align: left;"| | + | | style="text-align: left;"| Some error messages are not precise with bad alignment. |
− | + | | style="text-align: left;"| Refine all the error messages. | |
− | | style="text-align: left;"| | ||
− | |||
|} | |} | ||
</center> | </center> | ||
Line 408: | Line 415: | ||
<p> | <p> | ||
<ul> | <ul> | ||
− | <li> | + | <li>All test cases are passed. </li> |
− | <li> | + | <li>The introduction of intent matching affects the accuracy of Chatbot. </li> |
</ul> | </ul> | ||
</p> | </p> |
Revision as of 21:22, 8 October 2018
Contents
Internal Testing 1
Testing Plan
- Date: 6 July 2018
- Iteration: 3
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objective
- To test the accuracy rate of the chatbot
Scope
Chatbot |
---|
Portnet Related Enquiries |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 85% | Develop more testing questions to expand the testing coverage for next round of testing |
Overall Result
- The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.
Internal Testing 2
Testing Plan
- Date: 26 July 2018
- Iteration: 4
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
- To ensure the integration for chatbot and dashboard is successful
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Dashboard - Hits Analysis | |
Dashboard - Category Analysis |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 77% | The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call. |
Admin Portal | Dashboard is behaving as expected except minor error. | Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 3
Testing Plan
- Date: 4 Aug 2018
- Iteration: 5
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Enquiries | Dashboard - Hits Analysis |
API Module | Dashboard - Category Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - Bootstrap |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 77%
Minor error for API discovered, which is not critical. |
Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | Dashboard is behaving as expected.
Account management is behaving as expected except minor error. |
Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 4
Testing Plan
- Date: 5 Sept 2018
- Iteration: 7
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both Chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Dashboard - Filter by Time Frame | |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 70% | |
Admin Portal | Some error messages are not precise with bad alignment. | Refine all the error messages. |
Overall Result
- All test cases are passed.
- The introduction of intent matching affects the accuracy of Chatbot.
Internal Testing 3
Testing Plan
- Date: 4 Aug 2018
- Iteration: 5
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Enquiries | Dashboard - Hits Analysis |
API Module | Dashboard - Category Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - Bootstrap |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 77%
Minor error for API discovered, which is not critical. |
Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | Dashboard is behaving as expected.
Account management is behaving as expected except minor error. |
Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.