IS480 Team wiki: 2018T1 analyteaka UT2
Venue: Scanteak's Toh Guan Showroom - #01-22/23, Tradehub 21, 8 Boon Lay Way, 609964
Date: 05 Oct 2018
Time: 3:30pm – 5:30pm
Duration: 120 minutes
Number of Participants: 5
Objectives/ Business Case
Why are we doing this test ?
- Gather feedback on developed user interface and developed functions. Major change in UI for Store and Customer module.
- Identify potential usability issues
- Gather comments for future improvement
What are the benefits of doing this test?
- Improve overall user experience
- Ensuring the dashboard answer the business question
What is the risk of not doing this test?
- The system will be of lower quality and doesn't meet the needs of the business.
- Not validating the result will also reduce the usefulness or accuracy of the system.
Scope/ Product Under Test
- Google IAP authentication mechanism
- Customer profile page
- Store profile page
- Marketing planning tools
- Data upload
Procedures Participants are to think aloud as they follow through the instructions given for the user testing. The team will be there to record down the participants. Participants are to leave their feedback at the end of their user testing. The team will conclude the user testing by asking the participants question with regards to their behaviours and thought process.
|1||Participants should be able to login using the provided email||Reached|
|2||Participants should be able to view and interact with sales analytics and answer which store has the highest profile and best selling item.||Reached|
|3||Participants should be able to view and interact with customer analytics and answer the type of customer that visit the store and item they are likely to buy.||Reached|
|4||Participants should be able to interact with the marketing planning tool and ca reate campaign for one of the weekends in November||Reached|
|5||Participants should be able to interact with the marketing planning tool and create campaign for Christmas period||Reached|
|6||Participants should be able to upload the files to the server and view the data immediately.||Reached|
|7||Rate the overall UI design? Scale of 1-5, 1 being bad, 5 being per fect .Goal is 3||AVG 4 - (4,4,5,3,4)|
|8||Rate the overall ease of use? Scale of 1-5, 1 being very difficult, 5 being very easy. Goal is 3.||AVG 3.8 - ( 4,4,3,4,4)|
|9||How useful is the dashboard ? On a scale of 1-5, 1 being bad, 5 being perfect. Goal is 3.||AVG 4.4 – (5,4,4,4,5)|
|10||How useful is the marketing planning tools? ? On a scale of 1-5, 1 being bad, 5 being perfect. Goal is 3.||AVG 4 - (4,4,3,5,4)|
|Functions||Users' Comments||Changes to be made|
|User Interface – store||User were able to quickly understand the store charts.||NA|
|User Interface - store||User were able to quickly filter out stores using the filter function or dragging||NA|
|User Interface - Customer||User were excited by the data provided for cluster demographics||NA|
|User interface - Customer||User were calculating the spending per customer cluster by comparing size and revenue||Add in revenue per customer by cluster|
|User interface - planner||User were slightly confused by what consequents and antecedent means||Change it to item 1 and item 2|
|User interface - planner||User were slightly confused by the repeating item||NA|
|User interface - Data upload||Data upload failure for 2013 due to the difference in data format||NA – 2013 is caused by old system, we won’t be using them anymore.|
OthersFeedback:User feel that 1) the tools can help to reduce planning time, marketing time and cost. 2) Improve the way/accuracy they plan their marketing campaigns with limited marketing budget. 3) Provide better training for new sales staff based on the data
|Subject||Users' Comments||Changes to be made|
|Function Refinement - Planner||User mention would be good to have the cluster’s name instead of number||Will allow user to name the cluster.|
|Function Refinement- Data Upload||Certain customer prefer to give the phone number as NRIC instead of actual NRIC. This affects the actual result||To do NRIC validation checks.|
|Function Refinement - Store||Reduce the default window of the purchase date to the last quarter||Set default window of purchase date to last quarter same for customer|
|Function Refinement - Store||Reduce 24 hours to store opening hours mainly 8am to 10pm||Reducing the purchase time duration|
|Function Refinement - Store & Customer||User mention would be good to have a map by store or customer on a separate page||To add in maps on a separate tab.|
|Function Refinement - customer||Knowing customer’s actual HDB size is not as important as knowing the type (landed, condo or HDB)||Change the cluster HDB size to housing type|
|Function Refinement - Planner||They prefer to see all pairs, even if there’s a very low confidence level||To show all possible pairs.|
- Certain customer prefer to give phone number as NRIC instead of actual NRIC.
- The concentration of customer staying in Bukit Timah matches with what they are marketing.
- The low density of customer staying in Yishun, Woodland area matches with the customer’s target.
- They may have lower number of customer aged 45, but they have a higher spending per sales.
- Parents are likely to place their child’s information for delivery verification instead of theirs.
- 4PM provide the highest sales volume.
- Customer are promotion driven, can test the theory by having promotion with item pair with low confidence.
Overall ,the UT is rather successful and they like what was being show. They feel that they have benefited much from the current system and improve their operation and marketing process.
However, there’s always room for improvement. As such, they will be feeding us with feedback over the next few weeks as they continue to use the system.
The next step is to develop plan on rolling out for other countries as well.