IS480 Team wiki:2018T1 FORD INTERNAL TESTING
Contents
Internal Testing 1
Testing Plan
- Date: 6 July 2018
- Iteration: 3
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objective
- To test the accuracy rate of the chatbot
Scope
Chatbot |
---|
Portnet Related Enquiries |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 85% | Develop more testing questions to expand the testing coverage for next round of testing |
Overall Result
- The natural language processing model meets the expectation, and the accuracy rate is satisfactory for our clients.
Internal Testing 2
Testing Plan
- Date: 26 July 2018
- Iteration: 4
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
- To ensure the integration for chatbot and dashboard is successful
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Dashboard - Hits Analysis | |
Dashboard - Category Analysis |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Accuracy rate: 77% | The Chatbot needs to be modified to better instruct the user of how to use the chatbot for API call. |
Admin Portal | Dashboard is behaving as expected except minor error. | Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 3
Testing Plan
- Date: 4 Aug 2018
- Iteration: 5
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Enquiries | Dashboard - Hits Analysis |
API Module | Dashboard - Category Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - Bootstrap |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 77%
Minor error for API discovered, which is not critical. |
Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | Dashboard is behaving as expected.
Account management is behaving as expected except minor error. |
Slight changes of UI design may be proposed after a demonstration with the client. |
Overall Result
- With more testing question developed, the chatbot accuracy rate dropped.
- This is expected and the accuracy rate is still within a satisfactory range.
Internal Testing 4
Testing Plan
- Date: 5 Sept 2018
- Iteration: 7
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both Chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Dashboard - Filter by Time Frame | |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 70% | |
Admin Portal | Some error messages are not precise with bad alignment. | Refine all the error messages. |
Overall Result
- All test cases are passed.
- The introduction of intent matching affects the accuracy of Chatbot.
Internal Testing 5
Testing Plan
- Date: 1 Oct 2018
- Iteration: 9
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel Status | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Feedback | Dashboard - Top Question Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management | |
Feedback Management | |
Announcement Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 80% | Welcome message for the Chatbot need to be modified to better instruct the user of how to use the Chatbot for API call. |
Admin Portal | 1. Poor user experience for feedback function is detected. Eg. When a user submits a feedback, no confirmation message is displayed, the feedback window simply closed. Minor bugs for feedback display on the admin side is found.
2. Minor issue for timestamp display |
1. Success message should be displayed after the user submits a feedback.
2. Round up timestamp display to full number. |
Overall Result
- Overall good performance with one test case failed.
Internal Testing 6
Testing Plan
- Date: 1 Nov 2018
- Iteration: 12
- Location: SMU SIS GSR 2.3
- Testing system: Windows
- Testing platform: Google Chrome
Objectives
- To test whether the deployment is successful
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the Chatbot
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Check Container/Vessel Status | Dashboard - Hits Analysis |
Check for User Guide | Dashboard - Category Analysis |
Feedback | Dashboard - Top Question Analysis |
Account Management - Change Password | |
Account Management - Add New User | |
Data Management - File Upload | |
Data Management - FAQ Management | |
Feedback Management | |
Announcement Management |
Test cases & Test results
- Test cases & Test results can be found here
Key Findings
Functionality | Application Behaviour | Changes proposed |
---|---|---|
Chatbot | Test accuracy rate: 80%. When HTTPS is implemented, the application behaviour slightly varies.Unsafe script need to be loaded for the application to fully function. | Try to find free CA signed HTTPS certificate. |
Admin Portal | Bug found during to question index issue | Improve the database structurer. |
Overall Result
- Overall good performance with one test case failed.