Difference between revisions of "IS480 Team wiki: 2018T1 Five Stones User Testing UAT3"
Jump to navigation
Jump to search
Hyyeo.2016 (talk | contribs) (UAT3 main content update) |
Hyyeo.2016 (talk | contribs) (UAT3 main content update v23- All Content) |
||
(6 intermediate revisions by the same user not shown) | |||
Line 49: | Line 49: | ||
<div style="font-family: helvetica; font-size:15px"> | <div style="font-family: helvetica; font-size:15px"> | ||
<p> | <p> | ||
− | <b>Venue: </b><br> | + | <b>Venue: </b>SingPost Centre, Coffee Bean<br> |
− | <b>Date: </b><br> | + | <b>Date: </b>11 November 2018, Sunday<br> |
− | <b>Time: </b><br> | + | <b>Time: </b>5 pm<br> |
− | <b>Number of Participant(s): </b><br> | + | <b>Number of Participant(s): </b>4<br> |
− | <b>Target Audience: </b><br> | + | <b>Target Audience: </b>Sponsor’s clients (Developers)<br> |
</p> | </p> | ||
</div> | </div> | ||
==Objectives== | ==Objectives== | ||
+ | <div style="font-family: helvetica; font-size:15px"> | ||
+ | <ul> | ||
+ | <li>To test core developed functionalities to gather feedback and gain deeper insights (i.e. Login, Creation of reports & exporting functions such as Powerpoint of PDF format)</li> | ||
+ | <li>To identify potential usability issues from users while using the web application</li> | ||
+ | <li>To gather comments following changes to UAT 2 for further improvements</li> | ||
+ | <li>To improve SCube web application’s overall User Interface (UI) and User Experience (UX)</li> | ||
+ | </ul> | ||
+ | </div> | ||
==Scope== | ==Scope== | ||
+ | <div style="font-family: helvetica; font-size:15px"> | ||
+ | <ol> | ||
+ | <li>Account Module<ul><li>Login/Logout</li></ul></li> | ||
+ | <li>Dashboard Module<ul><li>Displaying business reports with basic graphs</li></ul></li> | ||
+ | <li>Report Components module<ul><li>Draggable components (i.e. text, header, image, charts, video)</li></ul></li> | ||
+ | <li>Report Customization Module<ul><li>User-customizable options for component properties (e.g. formatting of text and charts)</li></ul></li> | ||
+ | <li>Importing module<ul><li>Process different data types from multiple data sources</li></ul></li> | ||
+ | <li>Chart Population Module<ul><li>Loading data into charts</li><li>Integrate retrieved data sources to allow customization of report components and properties</li></ul></li> | ||
+ | <li>Report Descriptive Analytics Module<ul><li>Display descriptive information of report (i.e. Summary Statistics on charts)</li></ul></li> | ||
+ | <li>Exporting Module<ul><li>Exporting created report to PPT format</li><li>Exporting created report to PDF format</li></ul></li> | ||
+ | <li>Slideshow Module<ul><li>View saved report as a slideshow in the browser</li></ul></li> | ||
+ | <li>Communications Module<ul><li>Support emailing of exported report to client’s email</li></ul></li> | ||
+ | <li>Report Preview Module<ul><li>Preview the report with component values according to URL parameter</li><li>Generate PDF based on preview</li></ul></li> | ||
+ | </ol> | ||
+ | </div> | ||
==Procedure== | ==Procedure== | ||
+ | <div style="font-family: helvetica; font-size:15px"> | ||
+ | <ol> | ||
+ | <li>Participants will be provided with the link to the web application</li> | ||
+ | <li>A set of task instructions will be will be handed out to participants</li> | ||
+ | <li>Participants are encouraged to think aloud when they do the testing</li> | ||
+ | <li>Participants will be given minimal assistance during the test so that user behavior can be observed by the team</li> | ||
+ | <li>There will be a scribe assigned by the team to record down user behaviors, comments and feedback from the participants</li> | ||
+ | <li>Participants will help the team fill in the post-UAT feedback form</li> | ||
+ | </ol> | ||
+ | </div> | ||
==Results== | ==Results== | ||
+ | {| class="wikitable" style="font-family: Helvetica; background-color:#FFFFFF; width: 750px;" align="center" | ||
+ | |- | ||
+ | |||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="10pt" | S/N | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="150pt" | Goals | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="60pt" | Achieved? | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="170pt" | % of participants able to complete the task | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 1 | ||
+ | |style="text-align: left;"| Participants should be able to login and logout | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: left;"| Participants should be able to create a data source and select the data configurations without any guidance from the facilitator | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: left;"| Participants should be able to create a report template based on the default template or using a new blank template | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 4 | ||
+ | |style="text-align: left;"| Participants should be able to create and customize charts (i.e. Bar and Line charts) on the dashboard | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 5 | ||
+ | |style="text-align: left;"| Participants should be able to see basic summary statistics when they create a chart (i.e. Both bar & line chart inclusive) | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 6 | ||
+ | |style="text-align: left;"| Participants should be able to export the created report template as PPT and PDF format respectively | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 7 | ||
+ | |style="text-align: left;"| Participants should be able to view the created report template as a slideshow on the browser | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 8 | ||
+ | |style="text-align: left;"| Participants should be able to be to send out the created report template through in-build email function | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 9 | ||
+ | |style="text-align: left;"| Participants should be able to open up a selected report template in PDF format by using a given URL link | ||
+ | |style="text-align: center;"| Yes | ||
+ | |style="text-align: left;"| Pass. 100% successful | ||
+ | |||
+ | |- | ||
+ | |} | ||
+ | |||
+ | ==User Behavior== | ||
+ | {| class="wikitable" style="font-family: Helvetica; background-color:#FFFFFF; width: 750px;" align="center" | ||
+ | |- | ||
+ | |||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="10pt" | S/N | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="120pt" | Module/Function | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="160pt" | User Behavior | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="160pt" | Changes to be made | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 1 | ||
+ | |style="text-align: left;"| Overall UI design | ||
+ | |style="text-align: left;"| The “triple line” shown on the user interface is misleading. User clicked on the button thinking that it is like a menu drop down | ||
+ | |style="text-align: left;"| Change the icon on the user interface main page to reflect that the button represents collapsible instead of menu drop down | ||
+ | |||
+ | (i.e. Change to +/- icon buttons instead) | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: left;"| Developer / Creating a datasource | ||
+ | |style="text-align: left;"| Developer users doing the test are able to understand what is required from them. All developer users do not require assistance to complete the testing | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: left;"| Report Customization | ||
+ | |style="text-align: left;"| Users were able to create a simple business report using the report template quickly without any assistance | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 4 | ||
+ | |style="text-align: left;"| Exporting / Export as PPT | ||
+ | |style="text-align: left;"| Users did not have issues navigating to the correct position of the web application to find the button required to export the report template as PPT | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 5 | ||
+ | |style="text-align: left;"| Exporting / Export as PDF | ||
+ | |style="text-align: left;"| Users did not have issues navigating to the correct position of the web application to find the button required to export the report template as PDF | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 6 | ||
+ | |style="text-align: left;"| Slideshow | ||
+ | |style="text-align: left;"| User did not notice the “Exit Slide Show” button and does not know that there the button exists to exit the slide show | ||
+ | |style="text-align: left;"| Improve colour scheme and the button highlighting to make it obvious for users | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 7 | ||
+ | |style="text-align: left;"| Communications / Email | ||
+ | |style="text-align: left;"| User went on to click on the “Attached” button thinking that the report template needs to be attached | ||
+ | |||
+ | (i.e. User did not realize that the files are already attached) | ||
+ | |||
+ | |style="text-align: left;"| Increase the font for the pre-attached report templates to highlight to users that the chosen report template has been attached | ||
+ | |||
+ | |- | ||
+ | |} | ||
+ | |||
+ | ==User Comments== | ||
+ | {| class="wikitable" style="font-family: Helvetica; background-color:#FFFFFF; width: 750px;" align="center" | ||
+ | |- | ||
+ | |||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="10pt" | S/N | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="120pt" | Module/Function | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="160pt" | Comments | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="160pt" | Changes to be made | ||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 1 | ||
+ | |style="text-align: left;"| Navigation bar | ||
+ | |style="text-align: left;"| User got confused regarding the wording “template” in the web application. User got confused and thought that “templates” meant the templates they could choose to create the report | ||
+ | |style="text-align: left;"| Change the wording of “template” to “reports” instead to reduce confusion | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: left;"| Account / Create a new User | ||
+ | |style="text-align: left;"| User suggested that if the password check fails for the initial “Password” field, the system should not allow the user to carry on to “Confirm Password” field | ||
+ | |style="text-align: left;"| To show a error pop-up and stop the user from writing on the “Confirm Password” if the initial password written does not pass the password checker | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: left;"| Developer / Creating a datasource | ||
+ | |style="text-align: left;"| Developer page is very straightforward. Easy to understand what is required to be configured | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 4 | ||
+ | |style="text-align: left;"| Developer / developer page | ||
+ | |style="text-align: left;"| The label and the textbox field are too closely aligned together. | ||
+ | |||
+ | (i.e. The labelling and their textbox fields are too closely aligned together) | ||
+ | |||
+ | |style="text-align: left;"| Increase the spacing between the field labels and their respective textboxes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 5 | ||
+ | |style="text-align: left;"| Report Components / Draggable | ||
+ | |style="text-align: left;"| The textbox component was not fully draggable. Only the grey areas can be dragged | ||
+ | |style="text-align: left;"| Change the cursor of the mouse from the arrow to the “hand” icon to signal area that is draggable | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 6 | ||
+ | |style="text-align: left;"| Report Customization | ||
+ | |style="text-align: left;"| User recommended for the implementation of a ‘Search’ feature for the report templates to allow the finding of the templates to be quicker | ||
+ | |style="text-align: left;"| To implement a search feature to find the report template by name | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 7 | ||
+ | |style="text-align: left;"| Exporting / Export PPT | ||
+ | |style="text-align: left;"| User found that exporting report as PPT is very straightforward. No issues locating and exporting as local copy | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 8 | ||
+ | |style="text-align: left;"| Exporting / Export PDF | ||
+ | |style="text-align: left;"| User also found that exporting report as PDF is equally straightforward. No issues locating and exporting as local copy | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 9 | ||
+ | |style="text-align: left;"| Communications / Email | ||
+ | |style="text-align: left;"| The “send email” button at the email main page is confusing | ||
+ | |style="text-align: left;"| Change naming convention on button to “Compose Email” instead | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 10 | ||
+ | |style="text-align: left;"| Communications / Email | ||
+ | |style="text-align: left;"| There is no address book included in the email and users have to type in the email addresses by hand. Very prone to errors due to manual typing | ||
+ | |style="text-align: left;"| NIL | ||
+ | |||
+ | |- | ||
+ | |} | ||
+ | |||
+ | ==Quantitative Finding== | ||
+ | {| class="wikitable" style="font-family: Helvetica; background-color:#FFFFFF; width: 750px;" align="center" | ||
+ | |- | ||
+ | |||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="5pt" | S/N | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="190pt" | Description of Task | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | No. of clicks measured (Tester 1) | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | No. of clicks measured (Tester 2) | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | No. of clicks measured (Tester 3) | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | No. of clicks measured (Tester 4) | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | Average No. of clicks | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | Benchmark | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="110pt" | Factored Benchmark (by 1.5) | ||
+ | ! style="color: #FFFFFF; background-color:#334156;" width="100pt" | Target Met? | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 1 | ||
+ | |style="text-align: left;"| Participants should be able to create a data source as a Developer | ||
+ | |style="text-align: center;"| 15 | ||
+ | |style="text-align: center;"| 17 | ||
+ | |style="text-align: center;"| 15 | ||
+ | |style="text-align: center;"| 18 | ||
+ | |style="text-align: center;"| <b><u>16.25</u></b> | ||
+ | |style="text-align: center;"| <b>13</b> | ||
+ | |style="text-align: center;"| <b><u>19.50</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: left;"| Participants should be able to create a basic report template | ||
+ | |style="text-align: center;"| 30 | ||
+ | |style="text-align: center;"| 26 | ||
+ | |style="text-align: center;"| 23 | ||
+ | |style="text-align: center;"| 27 | ||
+ | |style="text-align: center;"| <b><u>26.50</u></b> | ||
+ | |style="text-align: center;"| <b>23</b> | ||
+ | |style="text-align: center;"| <b><u>34.50</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: left;"| Participants should be able to find and change the charts data on the dashboard | ||
+ | |style="text-align: center;"| 20 | ||
+ | |style="text-align: center;"| 20 | ||
+ | |style="text-align: center;"| 15 | ||
+ | |style="text-align: center;"| 17 | ||
+ | |style="text-align: center;"| <b><u>18</u></b> | ||
+ | |style="text-align: center;"| <b>15</b> | ||
+ | |style="text-align: center;"| <b><u>22.5</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 4 | ||
+ | |style="text-align: left;"| Participants should be able to export the report template as PPT format | ||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: center;"| <b><u>2.50</u></b> | ||
+ | |style="text-align: center;"| <b>2</b> | ||
+ | |style="text-align: center;"| <b><u>4.5</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 5 | ||
+ | |style="text-align: left;"| Participants should be able to export the report template as PDF format | ||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: center;"| 2 | ||
+ | |style="text-align: center;"| <b><u>2.25</u></b> | ||
+ | |style="text-align: center;"| <b>2</b> | ||
+ | |style="text-align: center;"| <b><u>4.5</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 6 | ||
+ | |style="text-align: left;"| Participants should be able to view the report template as a slideshow presentation from the browser | ||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: center;"| 4 | ||
+ | |style="text-align: center;"| 4 | ||
+ | |style="text-align: center;"| 3 | ||
+ | |style="text-align: center;"| <b><u>3.50</u></b> | ||
+ | |style="text-align: center;"| <b>4</b> | ||
+ | |style="text-align: center;"| <b><u>6</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |style="text-align: center;"| 7 | ||
+ | |style="text-align: left;"| Participants should be able to find and send the created report template as PPT & PDF via email | ||
+ | |style="text-align: center;"| 7 | ||
+ | |style="text-align: center;"| 9 | ||
+ | |style="text-align: center;"| 8 | ||
+ | |style="text-align: center;"| 10 | ||
+ | |style="text-align: center;"| <b><u>8.50</u></b> | ||
+ | |style="text-align: center;"| <b>7</b> | ||
+ | |style="text-align: center;"| <b><u>10.5</u></b> | ||
+ | |style="text-align: center;"| Yes | ||
+ | |||
+ | |- | ||
+ | |||
+ | |- | ||
+ | |} | ||
+ | |||
+ | <div style="font-family: helvetica; font-size:15px"> | ||
+ | Overall Status: | ||
+ | <ul> | ||
+ | <li>All the task managed to meet the target below the factored benchmark</li> | ||
+ | </ul> | ||
+ | </div> | ||
+ | |||
+ | ==Post-Test Questionnaire== | ||
+ | [[File:Post-Test UAT3 pic1.PNG | 900px]] | ||
+ | [[File:Post-Test UAT3 pic2.PNG | 900px]] | ||
+ | [[File:Post-Test UAT3 pic3.PNG | 900px]] | ||
+ | |||
+ | ==Overall Comments== | ||
+ | <div style="font-family: helvetica; font-size:15px"> | ||
+ | <ul> | ||
+ | <li>Web application is very straightforward and easy to use</li> | ||
+ | <li>Users liked the password checker at the account page</li> | ||
+ | <li>Users prefer if the web application has a “Search” function for the report templates to make searching easier</li> | ||
+ | </ul> | ||
+ | </div> |
Latest revision as of 16:10, 16 November 2018
HOME |
Persona & Scenarios | Prototypes | Meeting Minutes | Diagrams | User Testing | Deliverables |
Contents
UAT 3 Test Plan
Venue: SingPost Centre, Coffee Bean
Date: 11 November 2018, Sunday
Time: 5 pm
Number of Participant(s): 4
Target Audience: Sponsor’s clients (Developers)
Objectives
- To test core developed functionalities to gather feedback and gain deeper insights (i.e. Login, Creation of reports & exporting functions such as Powerpoint of PDF format)
- To identify potential usability issues from users while using the web application
- To gather comments following changes to UAT 2 for further improvements
- To improve SCube web application’s overall User Interface (UI) and User Experience (UX)
Scope
- Account Module
- Login/Logout
- Dashboard Module
- Displaying business reports with basic graphs
- Report Components module
- Draggable components (i.e. text, header, image, charts, video)
- Report Customization Module
- User-customizable options for component properties (e.g. formatting of text and charts)
- Importing module
- Process different data types from multiple data sources
- Chart Population Module
- Loading data into charts
- Integrate retrieved data sources to allow customization of report components and properties
- Report Descriptive Analytics Module
- Display descriptive information of report (i.e. Summary Statistics on charts)
- Exporting Module
- Exporting created report to PPT format
- Exporting created report to PDF format
- Slideshow Module
- View saved report as a slideshow in the browser
- Communications Module
- Support emailing of exported report to client’s email
- Report Preview Module
- Preview the report with component values according to URL parameter
- Generate PDF based on preview
Procedure
- Participants will be provided with the link to the web application
- A set of task instructions will be will be handed out to participants
- Participants are encouraged to think aloud when they do the testing
- Participants will be given minimal assistance during the test so that user behavior can be observed by the team
- There will be a scribe assigned by the team to record down user behaviors, comments and feedback from the participants
- Participants will help the team fill in the post-UAT feedback form
Results
S/N | Goals | Achieved? | % of participants able to complete the task |
---|---|---|---|
1 | Participants should be able to login and logout | Yes | Pass. 100% successful |
2 | Participants should be able to create a data source and select the data configurations without any guidance from the facilitator | Yes | Pass. 100% successful |
3 | Participants should be able to create a report template based on the default template or using a new blank template | Yes | Pass. 100% successful |
4 | Participants should be able to create and customize charts (i.e. Bar and Line charts) on the dashboard | Yes | Pass. 100% successful |
5 | Participants should be able to see basic summary statistics when they create a chart (i.e. Both bar & line chart inclusive) | Yes | Pass. 100% successful |
6 | Participants should be able to export the created report template as PPT and PDF format respectively | Yes | Pass. 100% successful |
7 | Participants should be able to view the created report template as a slideshow on the browser | Yes | Pass. 100% successful |
8 | Participants should be able to be to send out the created report template through in-build email function | Yes | Pass. 100% successful |
9 | Participants should be able to open up a selected report template in PDF format by using a given URL link | Yes | Pass. 100% successful |
User Behavior
S/N | Module/Function | User Behavior | Changes to be made |
---|---|---|---|
1 | Overall UI design | The “triple line” shown on the user interface is misleading. User clicked on the button thinking that it is like a menu drop down | Change the icon on the user interface main page to reflect that the button represents collapsible instead of menu drop down
(i.e. Change to +/- icon buttons instead) |
2 | Developer / Creating a datasource | Developer users doing the test are able to understand what is required from them. All developer users do not require assistance to complete the testing | NIL |
3 | Report Customization | Users were able to create a simple business report using the report template quickly without any assistance | NIL |
4 | Exporting / Export as PPT | Users did not have issues navigating to the correct position of the web application to find the button required to export the report template as PPT | NIL |
5 | Exporting / Export as PDF | Users did not have issues navigating to the correct position of the web application to find the button required to export the report template as PDF | NIL |
6 | Slideshow | User did not notice the “Exit Slide Show” button and does not know that there the button exists to exit the slide show | Improve colour scheme and the button highlighting to make it obvious for users |
7 | Communications / Email | User went on to click on the “Attached” button thinking that the report template needs to be attached
(i.e. User did not realize that the files are already attached) |
Increase the font for the pre-attached report templates to highlight to users that the chosen report template has been attached |
User Comments
S/N | Module/Function | Comments | Changes to be made |
---|---|---|---|
1 | Navigation bar | User got confused regarding the wording “template” in the web application. User got confused and thought that “templates” meant the templates they could choose to create the report | Change the wording of “template” to “reports” instead to reduce confusion |
2 | Account / Create a new User | User suggested that if the password check fails for the initial “Password” field, the system should not allow the user to carry on to “Confirm Password” field | To show a error pop-up and stop the user from writing on the “Confirm Password” if the initial password written does not pass the password checker |
3 | Developer / Creating a datasource | Developer page is very straightforward. Easy to understand what is required to be configured | NIL |
4 | Developer / developer page | The label and the textbox field are too closely aligned together.
(i.e. The labelling and their textbox fields are too closely aligned together) |
Increase the spacing between the field labels and their respective textboxes |
5 | Report Components / Draggable | The textbox component was not fully draggable. Only the grey areas can be dragged | Change the cursor of the mouse from the arrow to the “hand” icon to signal area that is draggable |
6 | Report Customization | User recommended for the implementation of a ‘Search’ feature for the report templates to allow the finding of the templates to be quicker | To implement a search feature to find the report template by name |
7 | Exporting / Export PPT | User found that exporting report as PPT is very straightforward. No issues locating and exporting as local copy | NIL |
8 | Exporting / Export PDF | User also found that exporting report as PDF is equally straightforward. No issues locating and exporting as local copy | NIL |
9 | Communications / Email | The “send email” button at the email main page is confusing | Change naming convention on button to “Compose Email” instead |
10 | Communications / Email | There is no address book included in the email and users have to type in the email addresses by hand. Very prone to errors due to manual typing | NIL |
Quantitative Finding
S/N | Description of Task | No. of clicks measured (Tester 1) | No. of clicks measured (Tester 2) | No. of clicks measured (Tester 3) | No. of clicks measured (Tester 4) | Average No. of clicks | Benchmark | Factored Benchmark (by 1.5) | Target Met? |
---|---|---|---|---|---|---|---|---|---|
1 | Participants should be able to create a data source as a Developer | 15 | 17 | 15 | 18 | 16.25 | 13 | 19.50 | Yes |
2 | Participants should be able to create a basic report template | 30 | 26 | 23 | 27 | 26.50 | 23 | 34.50 | Yes |
3 | Participants should be able to find and change the charts data on the dashboard | 20 | 20 | 15 | 17 | 18 | 15 | 22.5 | Yes |
4 | Participants should be able to export the report template as PPT format | 3 | 2 | 2 | 3 | 2.50 | 2 | 4.5 | Yes |
5 | Participants should be able to export the report template as PDF format | 2 | 2 | 3 | 2 | 2.25 | 2 | 4.5 | Yes |
6 | Participants should be able to view the report template as a slideshow presentation from the browser | 3 | 4 | 4 | 3 | 3.50 | 4 | 6 | Yes |
7 | Participants should be able to find and send the created report template as PPT & PDF via email | 7 | 9 | 8 | 10 | 8.50 | 7 | 10.5 | Yes |
Overall Status:
- All the task managed to meet the target below the factored benchmark
Post-Test Questionnaire
Overall Comments
- Web application is very straightforward and easy to use
- Users liked the password checker at the account page
- Users prefer if the web application has a “Search” function for the report templates to make searching easier