IS480 Team wiki:2018T1 FORD USER TESTING
Revision as of 00:04, 15 November 2018 by Jianan.lu.2015 (talk | contribs)
UAT 1
Testing Plan
- Date: 6 Aug 2018
- Iteration: 5
- Location: PSA Help Desk Office
- Participants: 6 PSA help desk staff
- Testing system: Windows
- Testing browser: Google Chrome
Objectives
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
- To test whether the functionalities and user experience meet expectations of the clients
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
User Instruction Button | Dashboard (Hits Analysis, Category Analysis, Top Question Analysis) |
Portnet Related Enquiries | FAQ Database Modification on UI |
Provide User Guide | FAQ Database Filter by Category |
Check Vessel/Container Status | Announcement Management |
Feedback |
Test Instruction Form
Key Comments Raised by the Help Desk Staff
Chatbot | Chatbot logic need to be improved to handle some corner questions, such as specific error code. |
---|---|
Admin Portal | 1. Dashboard should be more informative, eg dual axis for hit analysis graph
2. Time frame filtering function should be added, which allows users to filter by day, week, year |
Overall Result
- The admin portal is fully functioning with 0 bug detected.
- There are minor logic errors found for chatbot, which will be fixed during next iteration.
UAT 2
Testing Plan
- Date:27 September 2018 - 1 October
- Iteration:9
- Location: PSA Help Desk Office
- Participants: 6 PSA help desk staff
- Testing system: Windows
- Testing browser: Google Chrome
Objectives
- To test all the developed functionalities for both Chatbot and dashboard
- To test the accuracy rate of the Chatbot
- To test whether the functionalities and user experience meet the expectations of the clients
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Provide User Guide | Dashboard (Hits Analysis, Category Analysis, Top Question Analysis) |
Check Vessel/Container Status | FAQ Database Modification on UI |
Feedback | FAQ Database Filter by Category |
Feedback Management |
Test Instruction Form
Key Comments Raised by the Help Desk Staff
Chatbot | 1. User guide should be opened in a new tab instead of the original page, since the original user guide page contains the chat history 2. Submission of the feedback should not refresh the chatbot page, which will clear the chat history 3. The Chatbot did not provide the answer for some of the testers’ queries |
---|
Overall Result
- The admin portal is fully functioning with 0 bug detected. Its performance meets the expectation of the clients.
- Minor user experience issues are identified for chatbot, and the accuracy rate can be further improved. These changes will be discussed and implemented during the following iterations.
UAT 3
Testing Plan
- Date:1 November - 15 November 2018
- Iteration:12
- Location: PSA Help Desk Office
- Participants: 6 PSA help desk staff
- Testing system: Windows
- Testing browser: Google Chrome
Objectives
- To test all the developed functionalities for both chatbot and dashboard
- To test the accuracy rate of the chatbot
- To test whether the functionalities and user experience meet the expectations of the clients
Scope
Chatbot | Admin Portal |
---|---|
Portnet Related Enquiries | Authentication |
Provide User Guide | Dashboard (Hits Analysis, Category Analysis, Top Question Analysis) |
Check Vessel/Container Status | FAQ Database Modification on UI |
Feedback | FAQ Database Filter by Category |
Feedback Management |
Test Instruction Form
Key Comments Raised by the Help Desk Staff
Chatbot | 1. User guide should be opened in a new tab instead of the original page, since the original user guide page contains the chat history 2. Submission of the feedback should not refresh the chatbot page, which will clear the chat history 3. The Chatbot did not provide the answer for some of the testers’ queries |
---|
Overall Result
- User Acceptance Test is still on going!