HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2015T1 Team Hexa User Test 4"

From IS480
Jump to navigation Jump to search
 
(21 intermediate revisions by 2 users not shown)
Line 91: Line 91:
 
*Sign up & Questionnaire
 
*Sign up & Questionnaire
 
*Financial Summary
 
*Financial Summary
*Financial Health Ratios & Analysis
 
 
*Milestones
 
*Milestones
 
*Budgeting
 
*Budgeting
Line 118: Line 117:
 
===Demographics of Users===
 
===Demographics of Users===
  
===Quantitative Results===
+
During our UT4, we asked our users about their age and employment background. This also helps verify that our users are within the target scope of users for our application
 +
<br>
 +
<center> [[Image:UT4 Age Range.png|500px]] [[Image:UT4 Employment Status.png|500px]] </center>
 +
<br>
 +
 
 +
===Web Quantitative Results===
 +
<br>
 +
For the tasks based questions, we asked users to follow a set of instructions based on the functionalities of the application, they would then answer the ease in which they were able to complete the tasks given for each functionality tested - each tasks would be rated between 1 (Very Difficult) to 5 (Very Easy)
 +
<br>
 +
 
 +
<div align="left" style="padding:20px 0 10px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Web) Tasks Based Results</div>
 +
<br>
 +
 
 +
<center> [[Image:UT4 Task Results(Web).png|1050px]] </center>
 +
<br>
 +
<b>Task 1</b>
 +
*<b>Task 1.1</b>: Complete Sign up & Questionnaire
 +
*<b>Task 1.2</b>: View your credit profile analysis
 +
<b>Task 2</b>
 +
*<b>Task 2.1</b>: Update your Assets financial records
 +
*<b>Task 2.2</b>: Update your Income & Expenses financial records
 +
<b>Task 3</b>
 +
*<b>Task 3.1</b>: Adding a new Insurance
 +
*<b>Task 3.2</b>: Adding a new Debt
 +
<b>Task 4</b>
 +
*<b>Task 4.1</b>: Adding a new Milestone
 +
*<b>Task 4.2</b>: Adding a contribution to your Milestone
 +
<b>Task 5</b>
 +
*<b>Task 5.1</b>: Setting up a Budget
 +
*<b>Task 5.2</b>: Adding an Expense
 +
<b>Task 6</b>
 +
*<b>Task 6.1</b>: Retirement Planning Calculator
 +
*<b>Task 6.2</b>: Loan Comparison and Calculator
 +
<b>Task 7</b>
 +
*<b>Task 7.1</b>: Add a new friend
 +
*<b>Task 7.2</b>: Commenting on a post
 +
*<b>Task 7.3</b>: Create a new post
 +
<br>
 +
<div align="left" style="padding:20px 0 9px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Web)Task Results Analysis (Web)</div>
 +
Most users were able to complete Task 1 through 7, there are some users who have issues with completing the tasks in social function. a check with their feedback reveals that for tasks 7.2 where we asked users to find our a specific user's post, they had difficulty in searching for it and had to resort to scrolling through the page to manually find it. Some users also thought that they could find friends through the 'friends' tab, and thought that the find users and friends tab should be combined.
 +
 
 +
<br>
 +
 
 +
 
 +
<div align="left" style="padding:20px 0 10px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Web) Usefulness of Functions</div>
 +
<br>
 +
 
 +
<center> [[Image:UT4 Function breakdown.png |800px]] </center>
 +
<br>
 +
5 out of 9 functions had our at minimum <b>75%</b> of our testers giving a score of 4 or more, with 3 other functions having a minimum of <b>68%</b> of testers giving a score of 4 or more.
 +
Based on the task-based user testing of each feature, users were asked about the utility of the functions based on their needs. Generally, our testers find that most of the functions are useful and applicable to their needs. Some feel that it addresses their need for a personal financial planning tool that they can track on a weekly/monthly basis. Only the social function was scored badly among our testers, as there are some users who felt that the social function does not feel of use to them, and some are concerned that the ability to let other users see their financial status may be a privacy issue or even a potential security issue. Even though I-Wealth does not use any sensitive data that may identify users or link sensitive financial information through their social function, these users are not comfortable with sharing of non-sensitive financial details inside a personal financial planning application.
 +
 
 +
===Mobile Quantitative Results===
 +
Similar to the Web Task-Based Testing, we asked our 10 mobile testers to follow a set of instructions based on the functionalities of the application, they would then answer the ease in which they were able to complete the tasks given for each functionality tested - each tasks would be rated between 1 (Very Difficult) to 5 (Very Easy)
 +
<Br>
 +
<div align="left" style="padding:20px 0 10px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Mobile) Tasks Based Results</div>
 +
<br>
 +
 
 +
<center> [[Image:UT4 Task Results(Mobile).png|1050px]] </center>
 +
<br>
 +
 
 +
<div align="left" style="padding:20px 0 10px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Mobile) Usefulness of Functions</div>
 +
<br>
 +
<b>Task 1</b>
 +
*<b>Task 1.1</b>: Complete Sign up & Questionnaire
 +
*<b>Task 1.2</b>: View your credit profile analysis
 +
<b>Task 2</b>
 +
*<b>Task 2.1</b>: Update your Assets financial records
 +
*<b>Task 2.2</b>: Update your Income & Expenses financial records
 +
<b>Task 3</b>
 +
*<b>Task 3.1</b>: Adding a new Milestone
 +
*<b>Task 3.2</b>: Adding a contribution to your Milestone
 +
<b>Task 4</b>
 +
*<b>Task 4.1</b>: Setting up a Budget
 +
*<b>Task 4.2</b>: Adding an Expense
 +
<b>Task 5</b>
 +
*<b>Task 5.1</b>: Add a new friend
 +
*<b>Task 5.2</b>: Commenting on a post
 +
*<b>Task 5.3</b>: Create a new post
 +
<br>
 +
<div align="left" style="padding:20px 0 9px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Mobile)Task Results Analysis</div>
 +
Similar to the Web results, users were generally able to complete Task 1 through 5 without much difficulty, there were some testers that had difficulty in completing the task 5.2, which was difficult for users to comment on a specific post as they had to scroll through a number of posts on the mobile interface in order to find the right one.
 +
<br>
 +
 
 +
<center> [[Image:UT4 Function breakdown (Mobile).png|800px]] </center>
 +
<br>
 +
<div align="left" style="padding:20px 0 9px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
(Mobile) Usefulness of Functions</div>
 +
<br>
 +
We asked our mobile testers their view of the mobile function utility with regards to their needs. Similar to our web results, our mobile testers felt that the mobile functions were useful , especially with regards to the cash expense tracker. Some users felt that this was useful in planning their budget and help limit their expenses. the social function was seen by some of our testers as not that useful, albeit for different reasons as compared to the web results for social. Testers felt that the UI for the social function can be further improved, as the current UI does not easily help in the completion of the social tasks.
 +
<br>
 +
 
 +
===Overall Quantitative Results===
 +
<br>
 +
<center> [[Image:UT4 Overall.png|1050px]] </center>
 +
<br>
 +
 
 +
After completing of the tasks and feedback based on each of the functions to be tested, our testers were asked about the Overall usability and functionality of the I-Wealth Application. Testers were asked to rate between 1 (Worst) to 5 (Best) for Usefulness of the application, as well as the User Experience of it.
 +
 
 +
In terms of overall usefulness, our application scored relatively well among our testers- with more than <b>80%</b> of our testers giving a score higher than 4.
 +
As a whole they feel that it has the required functions that makes it a good personal financial planning and tracking application. Similarly, the overall user experience of our application has been rated almost as well, -with at least <b>75%</b> of our testers giving a score of 4 or more. Many of our users appreciate the simple usability of most parts of the applications. There are still some minor process improvements to be made that were found by the testers.
 +
 
 +
<center> [[Image:Top 5 important features.png|800px]] </center>
 +
<br>
 +
 
 +
In addition to the above questions, we asked our users - among the features that our application offers, to pick the top 5 important features to them.
 +
as seen from the above graph, features that had at least 70% mentions among users were
 +
* Financial Summary (Assets, Liabilities, Income & Expenses)
 +
* Cash Expense Tracker (Budget)
 +
* Financial Tools (Retirement Calculator & Loan Calculator)
 +
 
 +
These coincides with our market research that there is a need for a one-stop application that offers these different functions under a single account, allowing them to track their finances from the web & Mobile
 +
 
 +
The survey also showed that our testers find the social function to be the least important of the features that I-Wealth has, and as explained by the tasks-based results, many are not comfortable with mixing private financial information with a social function
  
 
===Qualitative Feedback===
 
===Qualitative Feedback===
 +
<div align="left" style="padding:20px 0 10px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
Web Feedback</div>
 +
*Sign up/Questionnaire
 +
**First collapsible should be expanded, and at the top of the page you can have a progress bar (0/3 completed) so that users can still see that there are three sections to be completed. I like that there is user feedback (something like, profile saved successfully in green) whenever I click on the "save" button, and that the next collapsible section will automatically expand for me to continue. I like the UI for task 1.2, easy to see my credit analysis.
 +
*Financial Summary
 +
**There are a lot of things to click at.
 +
**perhaps there could be a central 'Save Profile' button instead of an individual one.
 +
**Would be good if the tab can minimize after successfully updating.
 +
*Managing Debts / Insurance
 +
**Accidentally clicked on Cancel after filling up all the fields. Maybe can have a "Confirm Cancel?" popup.
 +
*Milestones
 +
**Allow manual entry for the dates. Currently if you use phone to update, you'll need to manually scroll through the months until 5 years later.
 +
** Date Picker for completion
 +
*Budget
 +
**Date by default should set for today
 +
**Date Picker
 +
**would be good to not hide the budget tools by default
 +
*Financial Tools
 +
**Probably allow for term input unit to be in years/months. Otherwise users need to manually convert 40 months into years which will take a while.
 +
**Very informative! Simple and clear:D I like the red and green smiley emoticons, it made the description more obvious (i.e shows if the result is good or bad).
 +
*Social
 +
**allows to click back to add friend page after adding of friends
 +
**Provide a search option for posts -cant find posts
 +
**limit size of pictures uploaded
 +
**posts seem chunked together and messy
 +
*Others
 +
**Can have links to reliable external websites such as CPF rates, on relevant pages.
 +
**could use some instructions or tooltips on forms and what the forms do
 +
**Looks great. Very user friendly.
 +
**Less wordy, but understandable
 +
**Great job, improve on web browser speed
 +
**Generally ok
 +
<br>
 +
 +
<div align="left" style="padding:20px 0 10px 0; font-weight:bold; line-height: 0.3em; font-size:16px; color:#000000">
 +
Mobile Feedback</div>
 +
*Financial Summary
 +
**Buttons appear too close to each other. Perhaps adding a little margin will help.
 +
**Good to have a notification that says that I have saved my changes or auto direct back to home.
 +
*Social
 +
**difficult to find friend's post when flooded by other people's posts
 +
*Others
 +
**Margins for buttons. Clearer logout button.
 +
**Alerts when I make changes
 +
<br>
  
 
===UT4 Documentations===
 
===UT4 Documentations===
 
+
*[[Media:User Test 4 - Test Plan.pdf|Test Plan]]<br/>
 +
*[[Media:I-Wealth User Testing 4 (Web) - Google Forms.pdf|Web Test Survey Questions]]<br/>
 +
*[[Media:I-Wealth_User_Testing_4_(Mobile)_-_Google_Drive.pdf|Mobile Test Survey Questions]]<br/>
 +
*[[Media:User Test 4 - Results.pdf|Test Results]]<br/>
  
 
<!---------------END of content-------------------------->
 
<!---------------END of content-------------------------->

Latest revision as of 22:37, 22 November 2015

Cheng logo.png

HOME   ABOUT US   PROJECT OVERVIEW   PROJECT MANAGEMENT   DOCUMENTATION
TECHNICAL DOCUMENTS   DESIGN DOCUMENTS   TEST DOCUMENTS   MINUTES   PROJECT DOCUMENTS   DEPLOYMENT
INTERNAL TESTING USER TEST 1 USER TEST 2 USER TEST 3 USER TEST 4


User Test 4 (UT4) Test Plan

Objectives:

  • Gather feedback regarding user interface of developed functions
  • Detect usability issues based on user behaviour
  • Find out if new users are able to easily learn and use the application
  • Test success rate of users completing assigned tasks
  • Find out which developed functions are the most important & provides the best user experience
  • To use UT results to improve our Web Application / Mobile Application


Scope of Testing:

Web

  • Sign up & Questionnaire
  • Financial Summary
  • Financial Health Ratios & Analysis
  • Debt & Insurance
  • Milestones
  • Budgeting
  • Financial Tools
  • Social


Mobile

  • Sign up & Questionnaire
  • Financial Summary
  • Milestones
  • Budgeting
  • Social


Procedure

  • Pre User Testing
    • 10/11/15 – Send out emails/Message to the users briefing them about the aim of the user test and also giving them details of the actions they have to perform.


  • User Testing
    • 11/11/15 9:00 am - 18/11/15 01:00 am: Send out the URL of the website and the post-survey Google Form to the users. The users can login to the website from their desired location and browse through the website.


  • Post User Testing
    • 11/11/15 9:00 am - 18/11/15 01:00 am: The users can fill the post-survey questions from their desired location.
    • 19/11/15: Send out emails/Messages to the users to thank them for their contribution.
    • Consolidate the feedback gathered and notify the developers about issues.

User Test Results

The following are the quantitative results collated from the 32 unique user responses

  • Number of Web Users = 22
  • Number of Mobile Users = 10


Demographics of Users

During our UT4, we asked our users about their age and employment background. This also helps verify that our users are within the target scope of users for our application

UT4 Age Range.png UT4 Employment Status.png


Web Quantitative Results


For the tasks based questions, we asked users to follow a set of instructions based on the functionalities of the application, they would then answer the ease in which they were able to complete the tasks given for each functionality tested - each tasks would be rated between 1 (Very Difficult) to 5 (Very Easy)

(Web) Tasks Based Results


UT4 Task Results(Web).png


Task 1

  • Task 1.1: Complete Sign up & Questionnaire
  • Task 1.2: View your credit profile analysis

Task 2

  • Task 2.1: Update your Assets financial records
  • Task 2.2: Update your Income & Expenses financial records

Task 3

  • Task 3.1: Adding a new Insurance
  • Task 3.2: Adding a new Debt

Task 4

  • Task 4.1: Adding a new Milestone
  • Task 4.2: Adding a contribution to your Milestone

Task 5

  • Task 5.1: Setting up a Budget
  • Task 5.2: Adding an Expense

Task 6

  • Task 6.1: Retirement Planning Calculator
  • Task 6.2: Loan Comparison and Calculator

Task 7

  • Task 7.1: Add a new friend
  • Task 7.2: Commenting on a post
  • Task 7.3: Create a new post


(Web)Task Results Analysis (Web)

Most users were able to complete Task 1 through 7, there are some users who have issues with completing the tasks in social function. a check with their feedback reveals that for tasks 7.2 where we asked users to find our a specific user's post, they had difficulty in searching for it and had to resort to scrolling through the page to manually find it. Some users also thought that they could find friends through the 'friends' tab, and thought that the find users and friends tab should be combined.



(Web) Usefulness of Functions


UT4 Function breakdown.png


5 out of 9 functions had our at minimum 75% of our testers giving a score of 4 or more, with 3 other functions having a minimum of 68% of testers giving a score of 4 or more. Based on the task-based user testing of each feature, users were asked about the utility of the functions based on their needs. Generally, our testers find that most of the functions are useful and applicable to their needs. Some feel that it addresses their need for a personal financial planning tool that they can track on a weekly/monthly basis. Only the social function was scored badly among our testers, as there are some users who felt that the social function does not feel of use to them, and some are concerned that the ability to let other users see their financial status may be a privacy issue or even a potential security issue. Even though I-Wealth does not use any sensitive data that may identify users or link sensitive financial information through their social function, these users are not comfortable with sharing of non-sensitive financial details inside a personal financial planning application.

Mobile Quantitative Results

Similar to the Web Task-Based Testing, we asked our 10 mobile testers to follow a set of instructions based on the functionalities of the application, they would then answer the ease in which they were able to complete the tasks given for each functionality tested - each tasks would be rated between 1 (Very Difficult) to 5 (Very Easy)

(Mobile) Tasks Based Results


UT4 Task Results(Mobile).png


(Mobile) Usefulness of Functions


Task 1

  • Task 1.1: Complete Sign up & Questionnaire
  • Task 1.2: View your credit profile analysis

Task 2

  • Task 2.1: Update your Assets financial records
  • Task 2.2: Update your Income & Expenses financial records

Task 3

  • Task 3.1: Adding a new Milestone
  • Task 3.2: Adding a contribution to your Milestone

Task 4

  • Task 4.1: Setting up a Budget
  • Task 4.2: Adding an Expense

Task 5

  • Task 5.1: Add a new friend
  • Task 5.2: Commenting on a post
  • Task 5.3: Create a new post


(Mobile)Task Results Analysis

Similar to the Web results, users were generally able to complete Task 1 through 5 without much difficulty, there were some testers that had difficulty in completing the task 5.2, which was difficult for users to comment on a specific post as they had to scroll through a number of posts on the mobile interface in order to find the right one.

UT4 Function breakdown (Mobile).png


(Mobile) Usefulness of Functions


We asked our mobile testers their view of the mobile function utility with regards to their needs. Similar to our web results, our mobile testers felt that the mobile functions were useful , especially with regards to the cash expense tracker. Some users felt that this was useful in planning their budget and help limit their expenses. the social function was seen by some of our testers as not that useful, albeit for different reasons as compared to the web results for social. Testers felt that the UI for the social function can be further improved, as the current UI does not easily help in the completion of the social tasks.

Overall Quantitative Results


UT4 Overall.png


After completing of the tasks and feedback based on each of the functions to be tested, our testers were asked about the Overall usability and functionality of the I-Wealth Application. Testers were asked to rate between 1 (Worst) to 5 (Best) for Usefulness of the application, as well as the User Experience of it.

In terms of overall usefulness, our application scored relatively well among our testers- with more than 80% of our testers giving a score higher than 4. As a whole they feel that it has the required functions that makes it a good personal financial planning and tracking application. Similarly, the overall user experience of our application has been rated almost as well, -with at least 75% of our testers giving a score of 4 or more. Many of our users appreciate the simple usability of most parts of the applications. There are still some minor process improvements to be made that were found by the testers.

Top 5 important features.png


In addition to the above questions, we asked our users - among the features that our application offers, to pick the top 5 important features to them. as seen from the above graph, features that had at least 70% mentions among users were

  • Financial Summary (Assets, Liabilities, Income & Expenses)
  • Cash Expense Tracker (Budget)
  • Financial Tools (Retirement Calculator & Loan Calculator)

These coincides with our market research that there is a need for a one-stop application that offers these different functions under a single account, allowing them to track their finances from the web & Mobile

The survey also showed that our testers find the social function to be the least important of the features that I-Wealth has, and as explained by the tasks-based results, many are not comfortable with mixing private financial information with a social function

Qualitative Feedback

Web Feedback
  • Sign up/Questionnaire
    • First collapsible should be expanded, and at the top of the page you can have a progress bar (0/3 completed) so that users can still see that there are three sections to be completed. I like that there is user feedback (something like, profile saved successfully in green) whenever I click on the "save" button, and that the next collapsible section will automatically expand for me to continue. I like the UI for task 1.2, easy to see my credit analysis.
  • Financial Summary
    • There are a lot of things to click at.
    • perhaps there could be a central 'Save Profile' button instead of an individual one.
    • Would be good if the tab can minimize after successfully updating.
  • Managing Debts / Insurance
    • Accidentally clicked on Cancel after filling up all the fields. Maybe can have a "Confirm Cancel?" popup.
  • Milestones
    • Allow manual entry for the dates. Currently if you use phone to update, you'll need to manually scroll through the months until 5 years later.
    • Date Picker for completion
  • Budget
    • Date by default should set for today
    • Date Picker
    • would be good to not hide the budget tools by default
  • Financial Tools
    • Probably allow for term input unit to be in years/months. Otherwise users need to manually convert 40 months into years which will take a while.
    • Very informative! Simple and clear:D I like the red and green smiley emoticons, it made the description more obvious (i.e shows if the result is good or bad).
  • Social
    • allows to click back to add friend page after adding of friends
    • Provide a search option for posts -cant find posts
    • limit size of pictures uploaded
    • posts seem chunked together and messy
  • Others
    • Can have links to reliable external websites such as CPF rates, on relevant pages.
    • could use some instructions or tooltips on forms and what the forms do
    • Looks great. Very user friendly.
    • Less wordy, but understandable
    • Great job, improve on web browser speed
    • Generally ok


Mobile Feedback
  • Financial Summary
    • Buttons appear too close to each other. Perhaps adding a little margin will help.
    • Good to have a notification that says that I have saved my changes or auto direct back to home.
  • Social
    • difficult to find friend's post when flooded by other people's posts
  • Others
    • Margins for buttons. Clearer logout button.
    • Alerts when I make changes


UT4 Documentations