Difference between revisions of "IS480 Team wiki: 2016T1 Charlies-Angels-UAT2"
Xwzheng.2014 (talk | contribs) (Undo revision 270259 by Xwzheng.2014 (talk)) |
Xwzheng.2014 (talk | contribs) |
||
Line 299: | Line 299: | ||
| View Excel Report || 20 || 3 || 46.13 || 14 || 95 || Format Excel nicely please | | View Excel Report || 20 || 3 || 46.13 || 14 || 95 || Format Excel nicely please | ||
|- | |- | ||
− | | Delete Portfolio || 20 || 0 || | + | | Delete Portfolio || 20 || 0 || 43.94 || 1 || 15 || Confirmation box upon delete |
|- | |- | ||
|} | |} |
Revision as of 14:09, 26 September 2016
Technical Documents | Wireframes | Quality Assurance | Meeting Minutes | Deliverables |
Contents
Survey Findings
Key Findings from Survey
This survey is meant to gather sentiments towards the features of our application that is intended to be rolled out (i.e. Dashboard, Robot Advisor). The targeted audience in this survey are millenials whom have little or no investment experience.
Key Findings:
- Total no. of participants: 50
- We have a fair mix of composition of gender, with 64% males and 36% females
- All participants were aged between 20-29
- 96% of our participants were Chinese, and 4% were Malay
- 22% of our participants have began working full time, 76% have not began, and the remaining 2% are currently looking for jobs
- We discovered that 92% of participants place between $0-$500 monthly into investments, or 8% > $1000. Surprisingly, there seem to not be a 'moderate' range of $501- $1000
- 68% of participants have not began investment, while the other 32% being distributed across several ranges
- 96% of participants felt that a feature to consider CPF and Housing Loan would be a useful one in a portfolio management
Graphical breakdowns of the survey can be found below:
We have also took our initial design of the application, specifically for the Robot Advisor to the users, and have requested for their feedback regarding the User Interface and Experience. The reason why we narrowed down to specifically Robot Advisor as it is a never-before-seen feature in any portfolio management application. The following are screenshots of the design that was shown to the users:
Screenshots of CPF-Robot Advisor
Screenshot #1:
Screenshot #2:
Screenshot #3:
User Feedback via Survey
Key Comments:
- Having to input so many fields at once is really troublesome. It is a turn off!
- I have not even began working, I don't have CPF, so effectively having me to enter 0s does not make sense
- I am not buying a house so soon, housing loan does not affect me
- Maybe a segmented survey would be more pleasing to the eyes
With these comments, we have properly evaluated that the user interface created was horrible and required immediate actions to be taken. This is despite of the good intentions of being thorough and precise when it came to calculations. Instead, we have considered what are the most essential of inputs, and have changed the user interface to ensure a more seamless experience for the user to utilise the feature.
Survey #2 Results
Survey #3 Results
Regression Testing
Results of Internal Unit Testing | ||
---|---|---|
Iteration No. | Date | Results |
3 | 06 JUL 2016 | 33/36; Bug Metrics of 7 - Hitting 1 High Impact & 2 Low Impact |
4 | 27 JUL 2016 | 69/74; Bug Metrics of 9 - Hitting 1 High Impact & 4 Low Impact |
5 | 10 AUG 2016 | 87/90; Bug Metrics of 7 - Hitting 1 High Impact & 2 Low Impact |
6 | 29 AUG 2016 | 101/105; Bug Metrics of 8 - Hitting 1 High Impact & 3 Low Impact |
7 | 8 SEP 2016 | 118/120; Bug Metrics of 6 - Hitting 1 High Impact & 1 Low Impact |
UAT #2 Results
Date: 15 Sep 2016 - 16 Sep 2016
Venue: School of Information Systems, Singapore Management University
Duration: 15 Sep 2016: 0900 - 1200 Hours @ SIS CR 3-2 & 1800 - 2100 Hours @ SIS CR 3-1; 16 Sep 2016: 0900 - 1200 & 1800 - 2100 Hours @ SIS CR 3-2
Number of Participants: 20
Target Audience: Millennials; aged 16-36 with Basic or No Investment Background
Objectives:
- Identify functionalities and usability issues
- Determine is the user interface is intuitive
- Discover ways to improve features and functions of the website based on the results of User Testing #2
- Introduce CPF & Housing Loan calculations - Never seen before feature in the market
Scope:
Functionalities tested are ones developed between Iteration 3 and 8
Procedure:
- A UAT document with instructions on how to test the website have been given to the testers
- Participants will log down the testing outcomes & team members would note the time taken per task in the sheet
- Participants are welcomed to type their general comments and feedback into the feedback textbox provided at the end of the test plan document
Tasks:
The UAT document requests participants to perform the following task(s):
To be filled in after UAT
For every task, participants were required to document down the following information:
- Execution of Task (Successful or Unsuccessful)
- Outcome
- Time Taken
- Login & Registration Test Cases
- Register a New Account
- Retrieve your Password
- Login
- Change your Password
- Portfolio Test Cases
- Create a Portfolio
- Search for a Stock
- Buy a Specific Stock
- Buy Two Other Stocks
- Sell One Stock
- PDF & Excel Test Cases
- View your Portfolio Summary
- Generate PDF Report of your Portfolio
- View your PDF Report summary and verify if the information is correct
- Generate Excel Report of your Portfolio
- View your Excel Sheet and verify if the information is correct
- Delete your Portfolio
- Watch List Test Cases
- Add any Two Stocks into your Portfolio
- Delete the stocks from your watch list
At the end of the test plan, we also asked the participants the following questions:
- Demographics Questions such as Age, Race, Work Experience, Investment Habits
- Do you have investment knowledge prior to using this application?
- Any Comments?
Demographics of our testers
- Total number of testers: 20
- Age group: 19 testers aged between 20 and 29; 1 tester aged above 30 years old
- Working: 3 testers are currently working
- Not Working: 17 testers are currently not working
- Investment Background: 12 of 20 testers indicated no background; 3 out of 20 testers indicated investment background of half a year; 3 out of 20 testers indicated investment background of 1 year; and the remaining 2 testers indicated investment background of 2 years
Login & Registration Test Cases
Function | No. of Testers | No. of Errors | Avg. Time (Sec) | Fastest (Sec) | Slowest (Sec) | Comments | |
---|---|---|---|---|---|---|---|
Registration | 20 | 0 | 36.75 | 16 | 56 | Username req too long | |
Retrieve Password | 20 | 0 | 32 | 7 | 118 | Email can be formatted better; Password that is resetted is too long; Let user see password field | |
Login | 20 | 0 | 12.31 | 2 | 41 | NIL | |
Change Password | 20 | 0 | 14.63 | 3 | 32 | Consider a confirm password field |
Portfolio Test Cases
Function | No. of Testers | No. of Errors | Avg. Time (Sec) | Fastest (Sec) | Slowest (Sec) | Comments |
---|---|---|---|---|---|---|
Create Portfolio | 20 | 1 | 11.63 | 6 | 22 | Can't create a portfolio called "test" but it appears in dropdown when I buy stocks |
Search for DBS Stock | 20 | 0 | 10.19 | 5 | 24 | NIL |
Buy DBS Stock | 20 | 0 | 18.38 | 6 | 33 | Page to show all stocks would be good; Buy/Sell like trading. Consider changing to add/remove. |
Buy 2 more Stocks | 20 | 0 | 58 | 2 | 126 | Populate last done price in modal & quantity too |
Sell any 1 Stock | 20 | 1 | 46.13 | 14 | 95 | Loading bar is too small; Confirmation box upon selling |
PDF & Excel Test Cases
Function | No. of Testers | No. of Errors | Avg. Time (Sec) | Fastest (Sec) | Slowest (Sec) | Comments |
---|---|---|---|---|---|---|
View Portfolio | 20 | 0 | 4.13 | 2 | 16 | NIL |
Generate PDF Report | 20 | 0 | 20 | 3 | 54 | Inform user that they already have clicked the PDF and show them the progress; Feature looks good. Very Useful; Might want to add disclaimer; Popup blocker will disable download, inform users if it is the case |
View PDF | 20 | 0 | 13 | 4 | 48 | Sold all Genting stocks but they are still under winning stocks; Trust/warrants not in the PDF. |
Generate Excel Report | 20 | 3 | 7 | 2 | 126 | Inform user that download has started; Icon looks like delete button |
View Excel Report | 20 | 3 | 46.13 | 14 | 95 | Format Excel nicely please |
Delete Portfolio | 20 | 0 | 43.94 | 1 | 15 | Confirmation box upon delete |
Watchlist Test Cases
Function | No. of Testers | No. of Errors | Avg. Time (Sec) | Fastest (Sec) | Slowest (Sec) | Comments |
---|---|---|---|---|---|---|
Add any 2 Stocks | 20 | 1 | 29.67 | 10 | 55 | NIL |
Delete the Stocks | 20 | 1 | 7.73 | 3 | 17 | Confirmation box upon delete |
No. | Risk Profiling |
---|---|
1 | A bit of doubt on accuracy of the app but good to have. Good functions. |
2 | Icon is misleading. Thought its settings icon; Setup during registration better for me; Put in profile should not be changed all the time. Some investors are greedy and wanna see rewards in short amt of time |
3 | Useful function; Put more questions for more accuracy |
4 | Form was ok |
5 | Assumes few questions to generate this > GOOD! Very good profiling. Questioning on accuracy of quarters |
6 | Robot advisor icon not clear. Did not know what that button is for. Breakdown is too general, because large cap stocks have different yield and prices |
7 | Good amt of questions. Trust that the report is accurate. Useful! |
8 | Good feature. Recommendation is fair enough. Default profile set looks suspicious |
9 | Decent. Consider as a pop-up to force reader to read. |
10 | Limited and some questions are similar. Q2 is too long. Breakdown is not clear. Change to another name! |
11 | include reasons for breakdown. Default not convincing. Change robot word to financial advisor. |
12 | Questions are okay. Function is useful. Can provide directions (what you should do) |
13 | -Remove default values, describe functionality instead and tell user what to do; Q2 too long, intimidating; Q1-3, 2-4, 3-2, 4-4, 5-1; Breakdown is very very very conservative for my input; Explain the terms like tooltip/link e.g. ETF |
14 | Useful and tell which things to buy. No. of questions are good. Not too long |
15 | Good feature. Questions just nice. |
16 | Q4 time range? Define it. Taken aback by form's default values. Should put robot vertically. |
17 | The default recommendation confused me |
18 | I like the allocation done, but some parts seem wordy. |
19 | Description abit wordy but otherwise its ok. |
20 | Question 2 needs to be rephrase a bit |
No. | Robot Advisor |
---|---|
1 | Good feature! Retirement analyser needs to have more output |
2 | Error should not stay so long; Increment and decrement makes no sense; Assumptions why got scroll bar; Describe that the est interest rate is automatically populated; Link with robot advisor? How to invest?; Give $ to earn as retirement goal |
3 | Form is ok, need to be more detailed; Can include additional salary per month, death age and future value at retirement. |
4 | Very useful. Inputs are ok ------- More details about retirement given would be helpful |
5 | Light bulb looks like an icon; Too many fields to fill in; Best if can use 1 accordion; Too many tabs is too troublesome for me; More details provided would be good, like telling me exactly how far I am away from my goal, maybe display a chart or something? |
6 | Good! Cpf and housing loan feature is fresh and very relevant; The application showed me how much I need to earn every month to reach my retirement goal. Its too vague and I don’t know what to do with the information! |
7 | Good feature, but input a lot of stuff and only getting 1 line of results is a bit disappointing |
8 | Sentencing structure is weird; Useful feature but questions is how it ties to business environment; Investment should be closer to how my portfolio is doing and advise |
9 | Useful to have; How would this affect my current investment goals? //might be a problem if they deleted their portfolio?
Would be good if there are clearer action steps for me. |
10 | Retirement nest value = 2 decimal places; if I can't hit the goals, what to do next? |
11 | Good function; Expected monthly salary? Or what salary?; Monthly salary based on monthy savings?; Make the function more prominent. It looks like fine print.
Would be good if you show me how exactly I can hit my retirement goals |
12 | Calculation might be wrong for retirement nest; Lack of info makes it unappealing. |
13 | Inflation rate put 4%, which is cpf rate. So usually investment must be more than 4%; Target pension term in sg context is it useful?; how much do you need per month (based on?); 2 decimal places; make assumptions tooltip more obvious, e.g. color; more explanation about retirement portion would be good |
14 | Very good. Information to earn is there; No. of questions is just nice. |
15 | A lot of inputs but will have a more accurate answer; Try to fit the inputs in a form, very lenghty; The amount of inputs I have to go through just to get 1 sentence of calculation does not seem worth it |
16 | -Expected salary? Entry level salary? ; Bank interest rate in savings acc is negligible; CPF details link to cpf site; Average salary income increment would be good; This feature is good. ; Feature is useful but too vague to be useful. |
17 | Telling me how to hit my retirement plans would be more useful than telling me to earn 10k a month for the next 40 years. |
18 | Feature is interesting, but lacks useful information. |
19 | NIL |
20 | Useful feature, but no advice was provided |
Overall Findings
No. | Overall Comment(s) |
---|---|
1 | Love it. Simple and intuitive to use. |
2 | Ui is not bad. Just the icon; Customize dashboard?; Homepage alignment; Filter for STI components; User icon beside username should always be there. |
3 | Simple ui. |
4 | Portfolio summary stocks have alert?; Easy to use, very clear ui |
5 | More data in graphs! But very good features so far |
6 | At dashboard the portfolio sector shows 2 stocks but I only have 1 stock; System is slow. Suggest to replace p/e, yield, p/b to ask/bid price; Fix decimal places. Show 2 decimal places; Portfolio robot advisor got error |
7 | Nil |
8 | UI easy and straightforward; Some icons needs to be clearer (robot advisor) |
9 | Dashboard can be neater! Make sure information and actors ar ebetter represented. |
10 | Make stockbook more obvious at the side nav bar; Search bar still contains stock after logging out. |
11 | Ui needs improvement. Keep up the good work. |
12 | Two scrollbar, horizontal bar not needed; Export pdf text should be made clickable. Design is nice. |
13 | Confirmation box for functions; Color and change text to remove after stocks is added; Information is straightforwrd, easy to find. Things are informative; Search bar is easy to find. Dashboard page quite long, need more organization; Ui is nice and smooth; Purple is weird for stock investment, does not feel professional; Confused as to targeted audience of app, some parts are filled with technical jargons while others feel like a guiding app; Robot advisor and retirement calculation is a good way to stand out; Try to add more value towards this direction since portfolio mgmt app is easily available. |
14 | UI is quite nice. Easy to navigate around the app. |
15 | UI cluttered; Under portfolio, icon seems out of place; No search results notification. Indicate that stocks are already in watchlist. |
16 | Filter equities to the top for search; UI is plain; Homepage feels weird. Icon should be linked to home page; STI component time looks off; Put div data in a table and change PE ratio to P/E ratio. Excel to have opening/closing price over a range of timing. |
17 | UI needs more graphs. A bit plain looking now. |
18 | Application looks promising but there are rooms for improvement. Retirement analyser portion really needs more info. If that is supposed to be your unique feature with a Singaporean element, make it more useful by providing useful info. |
19 | Keep up the good work! |
20 | Ui is simple and straightforward. Like it a lot. |
Key Improvements to be made
- Consider more visualizations
- Continue enhancing the User Interface
- Continue user-centric development as it is proving to be great!