IS480 Team wiki:2018T1 FORD INTERNAL TESTING
Revision as of 00:53, 21 August 2018 by Jianan.lu.2015 (talk | contribs)
Internal Testing 1
Testing Plan
- Date: 6 July 2018
- Iteration: 3
- Location: SMU SIS GSR 2.3
- Participants: 6 PSA help desk staff
- Testing system: Windows
- Testing platform: Google Chrome
Objective
- To test the accuracy rate of the chatbot
Scope
Chatbot |
---|
Portnet Related Enquiries |
Test cases & Test results
- Instructions can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 85% | Develop more testing questions to expand the testing coverage for next round of testing |
Overall Result
- The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.
Internal Testing 2
Testing Plan
- Date: 26 July 2018
- Iteration: 4
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
- To ensure the integration for chatbot and dashboard is successful
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Dashboard - Hits Analysis | |
Dashboard - Category Analysis |
Test cases & Test results
- Instructions can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 77% | The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call. |
Admin Portal | Dashboard is behaving as expected except minor error. | Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 3
Testing Plan
- Date: 4 Aug 2018
- Iteration: 5
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Enquiries | Dashboard - Hits Analysis |
API Module | Dashboard - Category Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - Bootstrap |
Test cases & Test results
- Instructions can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 77%
Minor error for API discovered, which is not critical. |
Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | Dashboard is behaving as expected.
Account management is behaving as expected except minor error. |
Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.