IS480 Team wiki: 2018T1 Five Stones User Testing UAT3
Revision as of 16:10, 16 November 2018 by Hyyeo.2016 (talk | contribs) (UAT3 main content update v23- All Content)
HOME |
Persona & Scenarios | Prototypes | Meeting Minutes | Diagrams | User Testing | Deliverables |
Contents
UAT 3 Test Plan
Venue: SingPost Centre, Coffee Bean
Date: 11 November 2018, Sunday
Time: 5 pm
Number of Participant(s): 4
Target Audience: Sponsor’s clients (Developers)
Objectives
- To test core developed functionalities to gather feedback and gain deeper insights (i.e. Login, Creation of reports & exporting functions such as Powerpoint of PDF format)
- To identify potential usability issues from users while using the web application
- To gather comments following changes to UAT 2 for further improvements
- To improve SCube web application’s overall User Interface (UI) and User Experience (UX)
Scope
- Account Module
- Login/Logout
- Dashboard Module
- Displaying business reports with basic graphs
- Report Components module
- Draggable components (i.e. text, header, image, charts, video)
- Report Customization Module
- User-customizable options for component properties (e.g. formatting of text and charts)
- Importing module
- Process different data types from multiple data sources
- Chart Population Module
- Loading data into charts
- Integrate retrieved data sources to allow customization of report components and properties
- Report Descriptive Analytics Module
- Display descriptive information of report (i.e. Summary Statistics on charts)
- Exporting Module
- Exporting created report to PPT format
- Exporting created report to PDF format
- Slideshow Module
- View saved report as a slideshow in the browser
- Communications Module
- Support emailing of exported report to client’s email
- Report Preview Module
- Preview the report with component values according to URL parameter
- Generate PDF based on preview
Procedure
- Participants will be provided with the link to the web application
- A set of task instructions will be will be handed out to participants
- Participants are encouraged to think aloud when they do the testing
- Participants will be given minimal assistance during the test so that user behavior can be observed by the team
- There will be a scribe assigned by the team to record down user behaviors, comments and feedback from the participants
- Participants will help the team fill in the post-UAT feedback form
Results
S/N | Goals | Achieved? | % of participants able to complete the task |
---|---|---|---|
1 | Participants should be able to login and logout | Yes | Pass. 100% successful |
2 | Participants should be able to create a data source and select the data configurations without any guidance from the facilitator | Yes | Pass. 100% successful |
3 | Participants should be able to create a report template based on the default template or using a new blank template | Yes | Pass. 100% successful |
4 | Participants should be able to create and customize charts (i.e. Bar and Line charts) on the dashboard | Yes | Pass. 100% successful |
5 | Participants should be able to see basic summary statistics when they create a chart (i.e. Both bar & line chart inclusive) | Yes | Pass. 100% successful |
6 | Participants should be able to export the created report template as PPT and PDF format respectively | Yes | Pass. 100% successful |
7 | Participants should be able to view the created report template as a slideshow on the browser | Yes | Pass. 100% successful |
8 | Participants should be able to be to send out the created report template through in-build email function | Yes | Pass. 100% successful |
9 | Participants should be able to open up a selected report template in PDF format by using a given URL link | Yes | Pass. 100% successful |
User Behavior
S/N | Module/Function | User Behavior | Changes to be made |
---|---|---|---|
1 | Overall UI design | The “triple line” shown on the user interface is misleading. User clicked on the button thinking that it is like a menu drop down | Change the icon on the user interface main page to reflect that the button represents collapsible instead of menu drop down
(i.e. Change to +/- icon buttons instead) |
2 | Developer / Creating a datasource | Developer users doing the test are able to understand what is required from them. All developer users do not require assistance to complete the testing | NIL |
3 | Report Customization | Users were able to create a simple business report using the report template quickly without any assistance | NIL |
4 | Exporting / Export as PPT | Users did not have issues navigating to the correct position of the web application to find the button required to export the report template as PPT | NIL |
5 | Exporting / Export as PDF | Users did not have issues navigating to the correct position of the web application to find the button required to export the report template as PDF | NIL |
6 | Slideshow | User did not notice the “Exit Slide Show” button and does not know that there the button exists to exit the slide show | Improve colour scheme and the button highlighting to make it obvious for users |
7 | Communications / Email | User went on to click on the “Attached” button thinking that the report template needs to be attached
(i.e. User did not realize that the files are already attached) |
Increase the font for the pre-attached report templates to highlight to users that the chosen report template has been attached |
User Comments
S/N | Module/Function | Comments | Changes to be made |
---|---|---|---|
1 | Navigation bar | User got confused regarding the wording “template” in the web application. User got confused and thought that “templates” meant the templates they could choose to create the report | Change the wording of “template” to “reports” instead to reduce confusion |
2 | Account / Create a new User | User suggested that if the password check fails for the initial “Password” field, the system should not allow the user to carry on to “Confirm Password” field | To show a error pop-up and stop the user from writing on the “Confirm Password” if the initial password written does not pass the password checker |
3 | Developer / Creating a datasource | Developer page is very straightforward. Easy to understand what is required to be configured | NIL |
4 | Developer / developer page | The label and the textbox field are too closely aligned together.
(i.e. The labelling and their textbox fields are too closely aligned together) |
Increase the spacing between the field labels and their respective textboxes |
5 | Report Components / Draggable | The textbox component was not fully draggable. Only the grey areas can be dragged | Change the cursor of the mouse from the arrow to the “hand” icon to signal area that is draggable |
6 | Report Customization | User recommended for the implementation of a ‘Search’ feature for the report templates to allow the finding of the templates to be quicker | To implement a search feature to find the report template by name |
7 | Exporting / Export PPT | User found that exporting report as PPT is very straightforward. No issues locating and exporting as local copy | NIL |
8 | Exporting / Export PDF | User also found that exporting report as PDF is equally straightforward. No issues locating and exporting as local copy | NIL |
9 | Communications / Email | The “send email” button at the email main page is confusing | Change naming convention on button to “Compose Email” instead |
10 | Communications / Email | There is no address book included in the email and users have to type in the email addresses by hand. Very prone to errors due to manual typing | NIL |
Quantitative Finding
S/N | Description of Task | No. of clicks measured (Tester 1) | No. of clicks measured (Tester 2) | No. of clicks measured (Tester 3) | No. of clicks measured (Tester 4) | Average No. of clicks | Benchmark | Factored Benchmark (by 1.5) | Target Met? |
---|---|---|---|---|---|---|---|---|---|
1 | Participants should be able to create a data source as a Developer | 15 | 17 | 15 | 18 | 16.25 | 13 | 19.50 | Yes |
2 | Participants should be able to create a basic report template | 30 | 26 | 23 | 27 | 26.50 | 23 | 34.50 | Yes |
3 | Participants should be able to find and change the charts data on the dashboard | 20 | 20 | 15 | 17 | 18 | 15 | 22.5 | Yes |
4 | Participants should be able to export the report template as PPT format | 3 | 2 | 2 | 3 | 2.50 | 2 | 4.5 | Yes |
5 | Participants should be able to export the report template as PDF format | 2 | 2 | 3 | 2 | 2.25 | 2 | 4.5 | Yes |
6 | Participants should be able to view the report template as a slideshow presentation from the browser | 3 | 4 | 4 | 3 | 3.50 | 4 | 6 | Yes |
7 | Participants should be able to find and send the created report template as PPT & PDF via email | 7 | 9 | 8 | 10 | 8.50 | 7 | 10.5 | Yes |
Overall Status:
- All the task managed to meet the target below the factored benchmark
Post-Test Questionnaire
Overall Comments
- Web application is very straightforward and easy to use
- Users liked the password checker at the account page
- Users prefer if the web application has a “Search” function for the report templates to make searching easier