2012T1 Fortune Cookies: User Testings
Click here to go back to Fortune Cookies Progress Summary.
Contents
User Testing 1 (Heuristic Evaluation) - 1 August 2012
- to allow user to use early prototype product with the brief initial design of the flow.
- to match between client and Fortune Cookies' phrasing of words, contents, user interface, and functionalities in Delivery Excellence Dashboard.
Heuristic Evaluation gives users chance to experience the initial design of the application to be able to learn and discover about its functionalities, contents, etc.
Users will then give feedback which will be very helpful in term of understanding the strengths and weaknesses that current system has.
Tester:
- Michiel Roosjen
- listing the metric that users should be able to get , such as consistency, flexibility, and simplicity of design
- putting details on what users can expect on each metric
- ask user to give a scale and feedback for each metric
- create an action items list for us to work on
S/N | Description | Category | Status | Date Modified |
---|---|---|---|---|
1 | to be able to sort item | Functionality | done | 20 September |
2 | to grey out past project (only show current fiscal year) | User Interface | mitigated | - |
3 | to show history of previous years (according to manual close of project) | Content | mitigated | - |
4 | to have revenue and strategic importance to measure the level(high, medium, low) of project | Content | ... | ... |
5 | for admin to be able to arrange functions according to frequency of use | Functionality | done | 28 August |
6 | to have onsite checking of valid email and usernames | Functionality | done | 7 September |
7 | to order messages from top-down | Functionality | done | 20 September |
8 | to highlight error fields | Content | done | 7 September |
For Heuristic Evaluation, we need to work on functionalities and contents displayed by gathering more information and doing another user testing to find what client really wants.
Heuristic Report 01 - 1 August
User Testing 2 (Paper Prototype) - 31 August and 3 September 2012
click on Fortune Cookies: Paper Prototype Result
- Propose new and fresh ideas by exploring more layout design and functionalities to be implemented
- Get the preferable layout chosen by client
Paper prototype of improvements to the system is used to clarify requirements and enable draft interaction designs and screen designs to be very rapidly simulated and tested. Users are given the prototype to test the functionalities and usability of the whole system without any guidance. Reactions and doubts are recorded through video to note down the areas of improvements to be made in regards to usability and ease of use. Potential usability problems can be detected at a very early stage in the design process before any code has been written and User will get the feel of the comprehensive system and revise the requirements at an early stage.
Testers :
- Michiel Roosjen
- Rachel Tann
- Users are given a list of functionalities supported as discussed.
- Presented with the paper prototype, users will attempt to test the functionality/complete the tasks without any guidance.
- Tester will either voice out the interface the user should expect to see or add in prepared prototypes on other screenshots.
- Mark on the prototype where a user attempted to “click” or otherwise interact with the interface
S/N | Description | Category | Status | Date Modified |
---|---|---|---|---|
1 | To delete the status (red, yellow, or green bubble) when PM is filling in questionnaire. | Content | done | 9 September |
2 | to add more trigger to prompt user to answer questionnaire | Content | mitigated | - |
3 | to give "warning" to user before submitting the report | Content | ... | ... |
4 | to be able to generate report in Power Point | Functionality | ... | ... |
5 | to put the task list for project manager (the task only gets removed once the action has been completed), which will be put on top of the list of projects | Functionality | ... | ... |
6 | to add on option for turn on and off notification via email | Functionality | ... | ... |
7 | to implement the ascending sorting by 1st click and descending sorting by 2nd click | Functionality | done | 20 September |
8 | to have filter function, based on traffic light / status they are looking for (e.g. red) | Functionality | done | 21 September |
9 | to allow PM to edit project name | Functionality | done | 8 September |
10 | to remove the world map from edit region (only table left) | User Interface | N.A. | N.A. |
11 | to add back button from the individual project management page | Functionality | done | 21 September |
12 | to change the display for engagement partner. It will start with clicking on continent to country (and filter based on the status they want) and list of projects | Content | done | 14 September |
13 | to put the legend in the map to determine the severity and ease of viewing for partner | Content | done | 15 September |
14 | to include archives for past projects for all roles | Content | mitigated | - |
For paper prototype testing, all the action items are either functionality or content.
The team now is able to have a clearer view of what functionalities and contents that Lodestone wants for its Delivery Excellence Dashboard.
Paper Prototype Minutes 01 - 31 August
Paper Prototype Minutes 02 - 3 Sept (idem with external meeting minutes 08)
User Testing 3 (Hallway Test)- 18 September 2012
to check the test case , click on Hallway Test Case
- Explore if the display of data for current maps and chart is easily comprehensible/intuitive.
- Discover unsolved bugs in system.
- Gather feedbacks and ideas on improving current usability.
- Quick usability feedback on overview of the current prototype across a mix group of users.
- “A hallway usability test is where you grab the next person that passes by in the hallway and force them to try to use the code you just wrote. If you do this to five people, you will learn 95% of what there is to learn about usability problems in your code.”
Testers
- 15 students within SMU
- Approach random people with no background knowledge of the project.
- With a simple description of the application and a few general tasks and scenarios, user will attempt to complete it while exploring the application on his/her own.
- Bugs and difficulties are recorded by tester.
- User give feedback base on his/her experience.
S/N | Description | Category | Status | Date Modified |
---|---|---|---|---|
1 | To put story line and detailed steps on how to do certain tasks because testers are not familiar with the projects | N.A. | - | - |
2 | To provide more intuitive guidance by providing buttons in several pages
|
User Interface | done | 22 September |
3 | To debug project tab(error 404) | Functional | done | 21 September |
4 | To change date to calendar view instead of manually typing | User Interface | done | 22 September |
5 | To save questionnaire answer (currently if at least 1 questionnaire was not filled in, all the answers will be deleted and tester will need to fill in again the answer he/she previously keyed in) | Functional | done | 23 September |
6 | To put clearly which role user is having when they first signed in, it is not really obvious currently. | Content | ... | ... |
7 | System give null when Australia is chosen for add office. | Functional | done | 22 September |
User Testing 4 (Lodestone End User Test) - 25 September 2012
- To make sure that all agreeable functionalities between client and us are in the system
- To verify that all functions work properly
- Ensure that end users are able to perform their designated job functions with the new system
- Better understand the alignment of objectives and goals for each category of stakeholders
- Gather feedback on the quality and preference of data visualisation
End User Test is used to test whether all functionalities in the system work as they planned. By testing within the boundaries of work or task flows, End User test will be able to check if the intended output is available for user based on specific input given.
This test will involve 5 stakeholders who have different roles, which are :
- Stephen Wise - Engagement Partner
- Rui Imamura - Project Manager
- Ben Inglish - Project Manager
- ParameshwaranKN - Project Manager
- Rachel Tann - Admin
- Users will be given the test case for specific role assigned.
- Given the test case, users will follow the scenario, and put Pass / Fail in the column. Additional comments can also be added in by users.
S/N | Description | Category | Status | Date Modified |
---|---|---|---|---|
1 | To change back map to 3 basic colors instead of 9 | UI | ... | ... |
2 | To create 2 separate views for map dashboard for board members (optional) | Content | ... | ... |
3 | To change regions to: 1. North America |
Content | ... | ... |
4 | To remove Unnecessary Countries: New Cadelona, New Renuion and tiny islands at the top of Europe except for Iceland, Falkland, and Hawaii. | Content | ... | ... |
5 | To add Countries:Serbia, Hongkong/ Macao, and Luxembourg. | Content | ... | ... |
6 | To cut Russia into half in Secondary View | UI | ... | ... |
7 | To change popup to smaller stating that there is "No Project Running Currently" for countries that do not have projects running | Content | ... | ... |
8 | To change font color in stacked chart to black instead of grey and font size to be larger. | UI | ... | ... |
9 | New Projects happening in Brazil is not shown in Secondary View Chart but it showed up in Primary View Chart (All the countries in South America do not reflect the project no properly) | Functionality | ... | ... |
10 | Link between World and Regional Map is not working on IPAD. | Functionality | ... | ... |
11 | Heading in PM’s Dashboard should highlight what the bar chart stands for – Missing title | UI | ... | ... |
12 | To add Search Filter/ Multi Criteria Bar to filter search results | Functionality | ... | ... |
13 | to sort Overall Status of Project correctly. (currently Yellow > Red > Green) | Content | ... | ... |
14 | To separate Trend from Overall Status column to enable sorting for trend as well. | UI | ... | ... |
15 | To remove the extra cell of Project List in Firefox browser. | UI | ... | ... |
16 | To remove IDEA Phase | Content | ... | ... |
17 | To change "Past Performance" in chart to "Performance to Date” | UI | ... | ... |
18 | To be able to save draft | Functionality | ... | ... |
19 | To have a pop up to inform user whether he/she wants to extend the project end date or to close the project if project has past the end date. | Functionality | ... | ... |
20 | To give explanation of what strategic important is. | Content | ... | ... |
21 | To display add comment button for each pillar | Functionality | ... | ... |
22 | For Admin to be able to open back closed project | Functionality | ... | ... |
23 | To add a dropdown list to select currency. | Functionality | ... | ... |
24 | To change the chart to matrix for displaying of trend (optional) | Functionality | ... | ... |
25 | To include Read Receipt for Engagement Partner (optional) | Functionality | ... | ... |
26 | To remove NA from the status | Content | ... | ... |
27 | To resolve 'Pillars in Report Page are not sorted properly.' | Content | ... | ... |
28 | To add a preview page before submitting of new report. | Functionality | ... | ... |
29 | To add “date of submission” for new report submitted. | Functionality | ... | ... |
30 | To make text in comment box bigger. | UI | ... | ... |
31 | To be able to download report | Functionality | ... | ... |
32 | To disable back button when editing product | Functionality | ... | ... |
33 | To include email notification Engagement Partner after status report is submitted. | Functionality | ... | ... |
34 | To remove Office Name field | Content | ... | ... |
35 | To change Country drop-down list Auto-complete for Countries | Functionalities | ... | ... |
36 | To make the button states "Add Project" instead of the tiny icon | UI | ... | ... |
Click here to open
Test Case for End User Test
A compiled report of tasks discussed with Lodestone after End User Test
End User Test Summary
User Testing 5 (End To End Test) - TBC
- to verify functional, performance, and reliability requirements of our application.
- to make sure that all components within the application interact correctly.
End To End Test is similar to integration test ; this occurs after unit testing. Users, based on their roles, will be given a test case with a specific input and expected output given. Our End To End Test will use a Big Bang approach which means that this test will cover most of the developed modules that linked together.
- ...
- ...
S/N | Description | Category | Status | Date Modified |
---|---|---|---|---|
1 | ... | ... | ... | ... |
2 | ... | ... | ... | ... |
...