HeaderSIS.jpg

IS480 Team wiki: 2014T1 Happy Sounds User Test 2

From IS480
Revision as of 22:26, 30 November 2014 by Qiyang.tan.2012 (talk | contribs)
Jump to navigation Jump to search

Happysounds logo.png

HomeButton.png HOME   Glyphicons 042 pie chart.png PROJECT OVERVIEW   Calendaricon.png PROJECT MANAGEMENT   Tableicon.png DOCUMENTATION   Aboutus.png ABOUT US
DIAGRAMS   TESTING   PROTOTYPE   PRESENTATION MATERIALS
INTERNAL TESTING USER TEST 1 USER TEST 2 USER TEST 3

User Test 2 (UT2) Test Plan

Number of Users: 75 existing Carousell users & New users
Venue: At user's convenience
Date:04/10/2014 to 06/10/2014
Duration: 45 mins per user
Objectives:

  • Gather feedback regarding user interface of developed functions
  • Detect usability issues based on user behavior
  • Find out if new users are able to easily learn and use the application
  • Find out if Offer Management is effective for different groups of users
  • Find out which developed functions are the most important & provides the best user experience
  • To use UT results to improve our web application

Scope of Testing:

  • Browse Product Listings
  • Search Product
  • Filter Product
  • Search Users
  • View User Profile and Listings by User
  • View Product Details
  • View & Add Product Comment
  • Buy Now (Chat)
  • Offer Management (Chat to Buy / Filters / Sort / Search)

User Test 2 (UT2) Consolidated Results

The following are the consolidated UT2 results from the 75 users from this UT.

Breakdown of the 75 users

56 of the users are existing Carousell users and the remaining 19 are new users who have not used Carousell mobile application prior to the test.
In order to do an in-depth analysis of the 56 existing Carousell users, we categorized them into 2 groups, namely the Power Users and Moderate User. Users are categorized as Power User if they launched the Carousell mobile application more than 7 times a week. After grouping the users, there are a total of 17 power users and 39 moderate user.

HS UT2 usercategorization2.png


We also categorized our existing users into Buyer and seller. Out of the 56 existing users, 36 of them are buyers and the remaining 20 users are sellers.

HS UT2 usercategorization1.png

Average Rating for Functionality and User Experience (Mobile vs Web Application)

The 56 existing Carousell users were first asked to rate the Functionality and User Experience for the mobile and web application on a scale from 1 to 5, 1 being the worst and 5 being the best. New Carousell users were not asked this question as they do not have the experience with Carousell mobile application and will be unable to provide the rating for the mobile application.

Overall

From the 56 responses, we calculated the average rating based on the individual rating as shown below.

HS UT2 overall mobile web comparison.png

Both the average rating for functionality and user experience between web and mobile application are similar with the mobile application slightly edging out the web application in both aspects.
For functionality rating, mobile application got an average score of 3.9, which is higher than web application by a minor difference of 0.1. As for user experience rating, mobile application got an average rating of 3.9 and the difference is 0.2.
The result is encouraging as we are almost on par with the mobile application even though our web is not fully developed and we are only half-way through our project.

Overall User Experience Rating (Power vs Moderate User)

We then did a further analysis by comparing the user experience average rating between the power user and moderate user.

HS UT2 mobile Web powermoderate.png

From the results, the user experience rating for moderate users are similar with only a difference of 0.1.
As for the user experience rating of power users, the difference is 0.3, which is slightly higher than the moderate users. After analyzing the feedback from the users, we found out that power users felt that the web application design needs to be further polished as some of them felt that unnecessary scrolling could be minimized in the landing page and white space could be better utilized.

Overall Functionality Rating (Power vs Moderate User)

We also compared the results for mobile and web functionality rating between the power user and moderate user.

HS UT2 mobile Web functionality powermoderate.png

From the analysis, we found out that power user rated web application slightly better than the mobile application. However, moderate users felt that web functionality loses out to mobile slightly. This might be because some mobile functionalities are not currently available on the web application as our project is still in progress.

Average Functionality Rating based on User Experience

Overall

Users were then asked to rate the 12 tested functionalities based on User Experience on a scale from 1(Worst) to 5 (Best). The average values for the functionality rating based on user experience ranges from 3.5 to 4.1.

HS UT2 overall individualfunctionality rating.png

The top 3 functionalities that offer the best user experience are Search Products, View Listings by user and View & Add Comments to Products. The other functionalities, with the exception of Search for Users, also had good average ratings for user experience as the values ranges from 3.9 to 4.1. Search for Users had an average rating of 3.5 and this is mainly due to the function being not intuitive as most users were unable to locate the function and were unaware that they were required to click on the search toggle to switch between product and user search. However, our focus on other functionalities is correct as the more interactive functions are more important and search product is the least important functionality

Comparison of individual functionality User Experience rating (Power & Moderate User)

We then compared the individual functionality rating based on the 3 user categories, Power User, Moderate User and new user.

HS UT2 individual functionality UE usergroup.png

From our analysis based on the comparison, we found out that power users like the user experience for Search Products and Offer Management (Search) as they felt that these 2 functionalities are effective to search for products during browsing and search for offers in the offer management page. User Experience for Search users was rated poorly across all 3 user groups which resulted in the low average rating for the functionality.

Comparison of individual functionality User Experience rating (Buyer & Seller)

We also did a further analysis for the individual functionality user experience between the buyers and sellers

HS UT2 individual functionality UE buyerseller.png

This analysis proved that our initial claim is correct as buyers like Browse Product Listings, Product Filters and Search while sellers like Offer Management. Sellers also generally rated Offer Management better than Buy Now (Chat) as compared to Buy Now (Chat), which buyers preferred more. This supports our claim that Offer Management gives a better medium to manage offers and trades.

Average Functionality Rating based on Importance

Overall

Users were also asked to rate the functionalities based on Importance on the same scale from 1(Worst) to 5 (Best).

HS UT2 individual functionality importance.png

Although the other functionalities were generally well rated, Offer Management functionalities were not impressive. This is because users are not familiar with the concept of Offer Management as it is a new feature that is distinct from the mobile application. Furthermore, users were not familiar with the product in the test account for this UT as the test account was pre-populated with offers. Therefore, it was harder for the users to appreciate the filtering and sorting options available for Offers Management.

Comparison of individual functionality Importance rating (Power & Moderate User)

A further analysis on the individual functionalities was done to compared the difference in rating between the power and moderate users.

HS UT2 individual functionality importance usergroups.png

From on the chart above, power users consistently rated offer management functionalities highly compared to the other user groups. This showed that they felt offer management will help them to effectively manage their offers on Carousell. The lower average rating for offer management in the overall importance average rating shown in the previous section was due to the other user groups. As seen from the graph, new users rated offers management functionalities the lowest across the 3 groups.

Comparison of individual functionality Importance rating (Buyer & Seller)

Similarly, we compared the difference in ratings between the buyer and seller to discover any trends in the ratings. We discovered that product filter was more highly rated for buyers than sellers. They felt that product filters will help them to find and browse product listings more effectively.

HS UT2 individual functionality importance buyerseller.png

Qualitative Feedback & Problems

  • Category pictures on landing page are too big and this resulted in too much unnecessary scrolling
  • Search toggle for product and user search is not intuitive as many users have trouble finding Search for User function
  • Some icons / elements are not meaningful and users do not know what it represents. For example, the “+”, “-“, “o” icons on Product Details and User Profile page
  • Users were unsure if login is required to be able to add comments on Product Details page
  • The text displayed when no listings are found (No listings found. Here are some recently liked products instead!) is not obvious
  • The button text “Buy Now” on Product Details page is inaccurate as the user has to negotiate the product price with the seller before purchasing the product
  • Unsure of how to use Offers Management (Filters)
  • Unable to like products and follow user

Recommendations

  • Able to edit product details
  • More specific product categories such as Accessories in For Her category
  • Reduce size of category pictures on landing page
  • Add in message (information) of icons/elements on hover
  • Improve on Search for Users
  • Able to like product on Browse page
  • Zoom-in feature for main product image on Product Details page

Follow-up actions (Changes to be made)

  • Include text (product / user) or change background color for Search toggle to make user search more obvious
  • Reduce size of category pictures on landing page to minimize unnecessary scrolling
  • Change product filters to make it more explicit and obvious
  • Change product filters text input to slider
  • Add message that indicate login is required for adding comments
  • Increase the spacing between the text “No listings found. Here are some recently liked products instead!”, and recommended product image cards
  • Change button text “Buy Now” to “Chat to Buy” on Product Details page
  • Login prompt to inform user that login is required to use certain features

Changes Before & After UT2

Changes made Before After
Changed Search toggle div background to make the toggle more obvious
HS UT1 searchbar after.png
HS ut2 after searchbar.png
"Buy Now" button on product details page to "Chat to Buy"
HS buynow before.png
HS buynow after.png
Reduced category image size on landing page to minimize unnecessary scrolling
HS category before.png
HS category after.png
Changed Product Filters buttons to make it more explicit and obvious to the user
HS UT2 filters before.png
HS UT2 filters after.png
Changed Product Filters text input to slider
HS UT2 filters input before.png
HS UT2 filters input after.png
Added message to indicate login is required to leave a comment
HS comments before.jpg
HS comments after.png
Increased spacing between “No listings found. Here are some recently liked products instead!” message and product cards
HS recentlyliked before.jpg
HS recentlyliked after.png


New Additions

Changes After Change
Added Login Prompt to inform unauthenticated users that login is required to use certain features (Like Product, Offers, Chat to Buy) HS UT2 LoginPrompt1.png HS UT2 LoginPrompt2.png HS UT2 LoginPrompt3.png

UT2 Documentations