Difference between revisions of "IS480 Team wiki: 2018T1 Kaki&Kin/Documentations/Testing"
(One intermediate revision by the same user not shown) | |||
Line 638: | Line 638: | ||
| style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Resolved | | style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Resolved | ||
| style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Delimit by fullstops | | style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Delimit by fullstops | ||
− | | style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | | + | | style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Refactored the methods to allow taking in multiple sentences as a single utterance |
| style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Li Xuan | | style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | Li Xuan | ||
| style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | 15-Nov-18 | | style="font-size:13px; font-family:'Lucida Sans Unicode', 'Lucida Grande', sans-serif !important;; background-color:#ffffff;" | 15-Nov-18 |
Latest revision as of 21:22, 20 November 2018
Acceptance User Testing
Overview | ||
---|---|---|
1. Conducted at KeyReply’s office on 14 August 2018 | ||
2. Four test subjects had to do a series of tasks on the prototype new system | ||
3. Feedback was recorded from a questionnaire | ||
4. Proposed system was rated as easier to use than current system | ||
Test Subjects from KeyReply | ||
1. CTO | ||
2. Chatbot Trainer | ||
3. Developer | ||
4. UI/UX Designer | ||
Task | Recommendation | Average Rating (1 - very hard, 6 very easy) |
Uploading a text file into the system | 1. More explanation of the types of accepted files
2. Show indication of upload progress |
4.25 |
Finding a generated intent | 1. Font too small
2. Hard to click 3. Graph lines small and redundant |
3 |
Editing a sentence related to an intent | 1. Should tell users that enter key can be used
2. Guide the users to the empty fields 3. Unclear that variations can be deleted |
4 |
Adding alternative sentences for an intent | 1. Should tell users that enter key can be used
2. Guiding text is too lengthy 3. No clear indication that fields are for variations 4. Have a button to ‘Add’ new fields |
3.75 |
Adding a chatbot response/answer to an intent | 1. Guiding text is too lengthy and unclear
2. Have a button to ‘Add’ answer 3. Not obvious where to input answer |
3.5 |
Mid-Terms User Testing
Overview | |||
---|---|---|---|
1. Conducted at KeyReply’s office on 2 October 2018 | |||
2. Majority of feedback were on UI & UX | |||
3. Feedback was recorded from a questionnaire | |||
4. Kaki&Kin’s solution is still rated as being easier to use than existing method | |||
Test Subjects from KeyReply | |||
1. CTO | |||
2. Manager, Engineering | |||
3. UI/UX Designer | |||
Activity / Review | Recommendation /
Critical Feedback |
Average Rating (1 - worst score 6 - best score) | Insights |
Upload file | 1. More explanation on accepted file formats
2. Progress bar instead of spinner |
6 - very easy | More guidance could be given on upload file types. The spinner does not provide enough information on upload progress |
Keyword accuracy as a representation of Intent | 1. Too vague | 2.67 - below average | Further improvements to be made on NLP |
Visualisation Structure | 1. Representation is unclear
2. Merge ‘View’ & ‘Manage Content’ tabs |
2.67 - below average | Improve the current visualisation by making it easier to understand |
Deleting an Intent | 1. Drag and drop is easy but affected by list length
2. Trash icon button would be easier 3. Intent name should be editable |
4.67 - easier than average | Current method will be rendered hard to use by a long list of intent. Only allowing deletion of intent without editing intent name is too restrictive |
Adding question variations and answers | 1. Cannot see which intents are filled
2. Not enough hints and instructions 3. ‘Enter’ & ‘Backspace’ usage could cause input usability problems |
2.67 - below average | Change method for input and deletion. Show in a clear manner yet-to-be-filled intents. |
‘Review’ tab | 1. Take note of limitations with large intent numbers
2. Consider having collapsible boxes 3. Colour scheme makes it a bit hard to read |
Unrated | Take note of UI/UX suggestions |
Finals User Acceptance Testing
Overview | |||
---|---|---|---|
1. Conducted at KeyReply’s office on 19 November 2018 | |||
2. Data set used was client's own private data | |||
3. Kaki&Kin’s solution is an improvement for both Workflow 1 & Workflow 2 | |||
Test Subjects from KeyReply | |||
1. CTO | |||
2. Chatbot Developer | |||
Workflow 1 | |||
1. Time spent processing a file with 400+ utterances takes around 3 hours manually. The AIR system by Kaki&Kin takes around an hour | |||
2. Process is less painful with the removal of manual typing and eyeballing | |||
3. General clustering gives good overall view of data | |||
4. Project shows good potential but is not yet ready for live use | |||
5. Splitting of intents & merging of intents is not yet ready but can be done using workarounds | |||
Workflow 2 | |||
1. Roughly the same time savings as Workflow 1 | |||
2. Process is less painful as user can now directly train on the chat logs instead of eyeballing | |||
3. Automatic reassignment of new variations into matching existing intents where possible is highly useful | |||
4. A potential issue is that new Intents created may be duplicates of existing intents |
Bug Metrics | |||
---|---|---|---|
Severity | Description | ||
Low Impact (1 pt) | Unimportant. Typo error or small UI alignment issues | ||
High Impact (5 pts) | The system runs. However, some non-critical functionalities are not working. | ||
Critical Impact (10 pts) | The system is down or is usable after a short period of time. We have to fix the bugs to continue. | ||
Mitigation Plan | |||
Cumulative Current Unresolved Points | Plan | ||
Points <=5 | Fix during buffer time | ||
5 < points<10 | Fix during planned debugging time | ||
Points >= 10 | Stop current development and resolve the bug immediately |
Sprint 1 - Sprint 5 (Acceptance)
Unresolved Bug Points: 0
General back-end/NLP | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
CORS concerns on backend that does not allow front end to connect | Backend | Aaron | 8-Aug-18 | High Impact | 5 | Resolved | Allow(*) for cors, removed subscription option, false for websockets only | Went down to KR to understand how they set up GraphQL | Li Xuan | 9-Aug-18 | 0 | ||
Deployment to heroku unsuccessful | Backend | Li Xuan | 10-Aug-18 | High Impact | 5 | Resolved | server.js not found | Need to commit js files, not ts and added a procfile | Li Xuan & Zachery | 13-Aug-18 | 0 | ||
Upload Module | |||||||||||||
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
GraphQL runs on run time | Frontend | Eugene | 16-Aug-18 | High Impact | 5 | Resolved | Codes were executed on runtime | Execute the 'Skip' method in export function of GraphQL | Aaron | 16-Aug-18 | 0 | ||
Error thrown when read text file contents | Backend | Eugene | 16-Aug-18 | High Impact | 5 | Resolved | The uploaded object was not read as a text object | JSON.stringify() | Aaron | 16-Aug-18 | 0 | ||
Upon reading file, variables were not updated | Frontend | Eugene | 16-Aug-18 | High Impact | 5 | Resolved | The GraphQL server skip() function was not used | Used the skip() function | Aaron | 16-Aug-18 | 0 | ||
GraphQL not receiving string Object in Query | Frontend | Eugene | 16-Aug-18 | High Impact | 5 | Resolved | The variable required was not converted into a JSON object | Convert the variable into JSON object first | Aaron | 16-Aug-18 | 0 | ||
Edit Visualisation (View Interface) | |||||||||||||
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
Error in data structure of flattened data returned | Frontend | Eugene | 14-Aug-18 | High Impact | 5 | Resolved | Parsing and converting to string for JSON | Fixed the issues in the JSON template | Zachery | 16-Aug-18 | 0 | ||
Error on JSON file used for testing | Frontend | Eugene | 14-Aug-18 | High Impact | 5 | Resolved | Wrong assignment of variable | Changed the assignment of variable | Zachery | 16-Aug-18 | 0 | ||
Manage Content Node | |||||||||||||
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
Table not updated | Frontend | Eugene | 14-Aug-18 | Critical Impact | 10 | Resolved | JSON object deep freezed. | Requires parsing and stringify of return object | Zachery | 16-Aug-18 | 0 |
Sprint 6 - Sprint 8 (Mid-Terms)
Unresolved Bug Points: 0
General back-end/NLP | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
Tree recursive structure not compatible with front-end | Both | Li Xuan | 25-Sep-18 | Critical Impact | 10 | Resolved | JSON data structure mismatch | Matched the JSON data structure | Li Xuan | 26-Sep-18 | 0 | ||
Edit Visualisation (View Interface) | |||||||||||||
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
Flatten tree algorithm loses some children intents | Frontend | Eugene | 25-Sep-18 | High Impact | 5 | Resolved | Recursion issue | Redo recursion code | Li Xuan & Gabriel | 26-Sep-18 | 0 | ||
Default template appears sometimes instead of generated intents | Frontend | Eugene | 2-Oct-18 | High Impact | 5 | Resolved | File has not finished uploading when the 'Continue' button was clicked. Therefore, the default template loaded instead. | Show a circle loading animation while the file is being uploaded and do not allow the 'Continue' button to be clicked until the upload has finished | Zachery | 2-Oct-18 | 0 | ||
Not every child on lowest level is shown | Frontend | Li Xuan | 2-Oct-18 | High Impact | 5 | Resolved | Recursion issue | Issue was linked to earlier issue where flattened tree algorithm lost children | Li Xuan & Gabriel | 26-Sep-18 | 0 | ||
Some intents are generated even without Most representative text and variations | Frontend | Eugene | 2-Oct-18 | High Impact | 0 | Resolved | Checking for index > 0 instead of >= 0 | Removed check for index and print everything | Zachary | 12-Oct-18 | 0 | ||
Manage Content Node | |||||||||||||
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
Tree does not populate with variations of the same category by default | Frontend | Eugene | 18-Sep-18 | Critical Impact | 10 | Resolved | Printing issue | Printing from the tagged sentences of each child | Zachery | 18-Sep-18 | 0 | ||
Editing 1 variation edits the entire parents tree | Frontend | Zachery | 18-Sep-18 | Critical Impact | 10 | Resolved | Error in method to find children intents | Resolved and ensured that the method to find children does not affect parent | Zachery | 20-Sep-18 | 0 | ||
Deleting 1 variation deletes some other variations as well | Frontend | Eugene | 25-Sep-18 | High Impact | 5 | Resolved | Array Slicing | Limiting the array slicing to 1 object | Zachery | 26-Sep-18 | 0 | ||
Unclickable first child | Frontend | Zachary | 2-Oct-18 | High Impact | 5 | Resolved | List issue | Changing :list="list" to :list="data.Selected" | Zachery | 12-Oct-18 | 0 |
Sprint 9 - Sprint 11 (Finals)
Unresolved Bug Points: 0
General back-end/NLP | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Bug Description | Frontend / Backend | Tester | Date Found | Severity | Points Allocation | Resolved? | Cause | Action | Debugged By | Debugged Date | Current Points | ||
Data Cleansing does not remove '-' that are the first character on line or last character on line | Backend | Eugene | 24-Oct-18 | Low Impact | 1 | Resolved | Regex issue | Changed the regex expression | Eugene | 28-Oct-18 | 0 | ||
Some lines merge during data cleansing | Backend | Eugene | 30-Oct-18 | High Impact | 5 | Resolved | Suspected incorrect Regex expression | Changed the Regex expression in method remove_surrounding_colon_http_chat | Eugene | 30-Oct-18 | 0 | ||
Entry of chatlog with more than one sentence will be split in workflow 1 | Backend | Li Xuan | 8-Nov-18 | Critical Impact | 10 | Resolved | Delimit by fullstops | Refactored the methods to allow taking in multiple sentences as a single utterance | Li Xuan | 15-Nov-18 | 0 |