Difference between revisions of "IS480 Team wiki:2018T1 FORD INTERNAL TESTING"
Jump to navigation
Jump to search
Line 417: | Line 417: | ||
<li>All test cases are passed. </li> | <li>All test cases are passed. </li> | ||
<li>The introduction of intent matching affects the accuracy of Chatbot. </li> | <li>The introduction of intent matching affects the accuracy of Chatbot. </li> | ||
+ | </ul> | ||
+ | </p> | ||
+ | </div> | ||
+ | |||
+ | == <div style="background: #FF7504; padding: 15px; font-weight: bold; line-height: 0.3em; font-size:18px; font-family:helvetica" ><font color= #FFFFFF>Internal Testing 5</font></div> == | ||
+ | |||
+ | === Testing Plan === | ||
+ | <div style="font-family: Verdana; "> | ||
+ | <p> | ||
+ | <ul> | ||
+ | <li><b>Date: </b>1 Oct 2018<br/></li> | ||
+ | <li><b>Iteration: </b>9<br/></li> | ||
+ | <li><b>Location: </b>SMU SIS GSR 2.3<br/></li> | ||
+ | |||
+ | <li><b>Testing system: </b> Windows<br/></li> | ||
+ | <li><b>Testing platform: </b> Google Chrome</li> | ||
+ | </ul> | ||
+ | </p> | ||
+ | </div> | ||
+ | |||
+ | === Objectives=== | ||
+ | <div style="font-family: Verdana;> | ||
+ | <p> | ||
+ | <ul> | ||
+ | <li>To test whether the deployment is successful</li> | ||
+ | <li>To test all the developed functionalities for both chatbot and dashboard</li> | ||
+ | <li>To test the accuracy rate of the Chatbot</li> | ||
+ | </ul> | ||
+ | </p> | ||
+ | </div> | ||
+ | |||
+ | === Scope === | ||
+ | <center> | ||
+ | {|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center" | ||
+ | |- | ||
+ | ! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Chatbot | ||
+ | ! style="color:#ecf0f1; background-color:#FF7504;" width="400pt" | Admin Portal | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| Portnet Related Enquiries | ||
+ | |style="text-align: center;"| Authentication | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| Check Container/Vessel Status | ||
+ | |style="text-align: center;"| Dashboard - Hits Analysis | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| Check for User Guide | ||
+ | |style="text-align: center;"| Dashboard - Category Analysis | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| Feedback | ||
+ | |style="text-align: center;"| Dashboard - Top Question Analysis | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Account Management - Change Password | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Account Management - Add New User | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Data Management - File Upload | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Data Management - FAQ Management | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Feedback Management | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| | ||
+ | |style="text-align: center;"| Announcement Management | ||
+ | |- | ||
+ | |||
+ | |} | ||
+ | </center> | ||
+ | |||
+ | === Test cases & Test results === | ||
+ | <div style="font-family: Verdana;"> | ||
+ | <p> | ||
+ | <ul> | ||
+ | <li>Test cases & Test results can be found [https://docs.google.com/spreadsheets/d/12Pow7ThbIItgcasHOp6Tn7_UL8jzzG0DBsQj_Ethd5c/edit?usp=sharing here] <br/></li> | ||
+ | </ul> | ||
+ | </p> | ||
+ | </div> | ||
+ | |||
+ | === Key Findings === | ||
+ | <center> | ||
+ | {|class="wikitable" style="font-family: Verdana; background-color:#FFFFFF; width: 1000px;" align="center" | ||
+ | ! style="color:#ecf0f1; background-color:#FF7504;" width="200pt" | Functionality | ||
+ | ! style="color:#ecf0f1; background-color:#FF7504;" width="300pt" | Application Behaviour | ||
+ | ! style="color:#ecf0f1; background-color:#FF7504;" width="500pt" | Changes proposed | ||
+ | |- | ||
+ | | style="text-align: center;"| Chatbot | ||
+ | | style="text-align: left;"| Test accuracy rate: 80% | ||
+ | | style="text-align: left;"| Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. | ||
+ | |- | ||
+ | | style="text-align: center;"| Admin Portal | ||
+ | | style="text-align: left;"| 1. Poor user experience for feedback function is detected. Eg. When a user submits a feedback, no confirmation message is displayed, the feedback window simply closed. Minor bugs for feedback display on the admin side is found. | ||
+ | 2. Minor issue for timestamp display | ||
+ | | style="text-align: left;"| 1. Success message should be displayed after the user submits a feedback. | ||
+ | 2. Round up timestamp display to full number. | ||
+ | |- | ||
+ | |} | ||
+ | </center> | ||
+ | |||
+ | === Overall Result === | ||
+ | <div style="font-family: Verdana; "> | ||
+ | <p> | ||
+ | <ul> | ||
+ | <li>Overall good performance with one test case failed. </li> | ||
</ul> | </ul> | ||
</p> | </p> |
Revision as of 23:52, 14 November 2018
Contents
Internal Testing 1
Testing Plan
- Date: 6 July 2018
- Iteration: 3
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objective
- To test the accuracy rate of the chatbot
Scope
Chatbot |
---|
Portnet Related Enquiries |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 85% | Develop more testing questions to expand the testing coverage for next round of testing |
Overall Result
- The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.
Internal Testing 2
Testing Plan
- Date: 26 July 2018
- Iteration: 4
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
- To ensure the integration for chatbot and dashboard is successful
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Dashboard - Hits Analysis | |
Dashboard - Category Analysis |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 77% | The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call. |
Admin Portal | Dashboard is behaving as expected except minor error. | Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 3
Testing Plan
- Date: 4 Aug 2018
- Iteration: 5
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Enquiries | Dashboard - Hits Analysis |
API Module | Dashboard - Category Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - Bootstrap |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 77%
Minor error for API discovered, which is not critical. |
Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | Dashboard is behaving as expected.
Account management is behaving as expected except minor error. |
Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 4
Testing Plan
- Date: 5 Sept 2018
- Iteration: 7
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both Chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Dashboard - Filter by Time Frame | |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 70% | |
Admin Portal | Some error messages are not precise with bad alignment. | Refine all the error messages. |
Overall Result
- All test cases are passed.
- The introduction of intent matching affects the accuracy of Chatbot.
Internal Testing 5
Testing Plan
- Date: 1 Oct 2018
- Iteration: 9
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel Status | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Feedback | Dashboard - Top Question Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management | |
Feedback Management | |
Announcement Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 80% | Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | 1. Poor user experience for feedback function is detected. Eg. When a user submits a feedback, no confirmation message is displayed, the feedback window simply closed. Minor bugs for feedback display on the admin side is found.
2. Minor issue for timestamp display |
1. Success message should be displayed after the user submits a feedback.
2. Round up timestamp display to full number. |
Overall Result
- Overall good performance with one test case failed.
Internal Testing 5
Testing Plan
- Date: 1 Oct 2018
- Iteration: 9
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel Status | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Feedback | Dashboard - Top Question Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management | |
Feedback Management | |
Announcement Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 80% | Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | 1. Poor user experience for feedback function is detected. Eg. When a user submits a feedback, no confirmation message is displayed, the feedback window simply closed. Minor bugs for feedback display on the admin side is found.
2. Minor issue for timestamp display |
1. Success message should be displayed after the user submits a feedback.
2. Round up timestamp display to full number. |
Overall Result
- Overall good performance with one test case failed.