HeaderSIS.jpg

IS480 Team wiki: 2012T1 One-hit Wonder Final Wiki Usability Study

From IS480
Revision as of 22:32, 5 December 2012 by Kaili.tee.2009 (talk | contribs)
Jump to navigation Jump to search

HomeOHW.png Home

Project Progress

Project Management

Project Quality

Usability Study

Reflection

Slides and Poster

Usability Study: User Testing 2

Usability Study Recap

Usability Study Recap

HEURISTIC EVALUATION

In order to find out the usability of Flauntpage, we conducted a Heuristic Evaluation.

This HE focuses mainly on 8 Usability Heuristics focus points as follows:

  1. Be consistent
  2. Focus on tasks
  3. Provide feedback
  4. Follow established conventions
  5. Speak the user’s language
  6. Prevent errors
  7. Minimize clicks
  8. Avoid scrolling

Refer to detailed analysis of Heuristic Evaluation here


USER TESTING 1

Our first User Testing was conducted to gather more in-depth insights on the usability of FlauntPage and to ensure that the deployed version is running smoothly.

We also gathered many valuable feedbacks on how we could improve the workflow due to varying human behavior and other user interface improvements


Refer to detailed analysis of UT1 here


Difference between UT1 & UT2

UT1 UT2
Objectives To ensure that FlauntPage is performing well on the server with no major bugs and errors caught in the process. To find out the effectiveness of FlauntPage and how convenient it is for users to create employment documents using our application as compared to other tools like Microsoft Word.

To find out the effectiveness and value of FlauntPage and how convenient it is for companies to find suitable candidates as compared to other channels (e.g Other job portals, newspaper, company website etc)

Execution Guided.

Users assigned a role and given detailed test cases with guided steps to follow. Users rated difficulty and gave comments for each task completed.

Unguided.

Part I: Users assigned a role and given a resume template to create a replica of the resume using Microsoft words and FlauntPage. Timings will taken for creation of documents and we later observed users behavior when using FlauntPage.
Part II: A new group of testers to measure the quality of the resumes created by Part I's users.

UT2 Objective

UT2 Objective

Execution Period: 5th November 2012 - 7th November 2012 (3 days)


Job Seeker:

To find out the effectiveness of FlauntPage and how convenient it is for users to create employment documents using our application as compared to other tools like Microsoft Word.

Job Seeker
Total no. of testers 16

M.png M.png M.png M.png M.png M.png M.png M.png
F.png F.png F.png F.png F.png F.png F.png F.png

Browser used Google Chrome & Firefox
Age range 18 - 30
No. of testers assigned difficult level 10
No. of testers assigned easy level 6


Company:

To find out the effectiveness and value of FlauntPage and how convenient it is for companies to find suitable candidates as compared to other channels (e.g Other job portals, newspaper, company website etc)

Company
Total no. of testers 2
Browser used Google Chrome
Company name/Tester Richemont Luxury (Singapore) Pte Ltd and Hiring Consultant

Procedure

Procedure

Part 1:

  1. Tester will be assigned to complete a resume using either the easy or difficult version of the sample resume information
  2. Create a resume using Microsoft Word (Activity timed)
  3. Testers are required to format the layout and imagine that they are sending this resume out to apply for a job
  4. Login to dev.flauntpage.com and create a resume using the same piece of information provided (Activity timed)
  5. Team will record the time taken in completing the resume for both the MS Word and Flauntpage version.
  6. Navigate around the application – Flauntpage freely to try out other functionalities
  7. Answer a few questions on their general feelings of our application – FlauntPage


Download sample of easy template here
Download sample of difficult template here
Download sample of user overall experience feedback here


Part 2:

1. With all the resumes collected from the previous testers, we will get another group of testers to measure the quality of the resume. (Total 9)

  • 3 Testers were given 6 sets of EASY resumes (consist of 3 EASY MS Word & 3 EASY FlauntPage version, randomized)
  • 3 Testers were given 10 sets of DIFFICULT resumes (consist of 5 DIFFICULT MS Word & 5 DIFFICULT FlauntPage version, randomized)
  • 3 Testers were given 16 sets of EASY & DIFFICULT resumes (consist of 3 EASY MS Word & 3 EASY FlauntPage, 5 DIFFICULT MS Word & 5 DIFFICULT FlauntPage version, randomized)

2. Testers were then tasked to rate each resume between the scale of 1 to 5 (1 being the lowest)

  • They were asked to find their base ratings out of the pile first in order to ensure that subsequent ratings will be consistent

Results & Analysis

Results & Analysis

1. Based on the feedbacks provided by the testers in Part 1 of the testing, team identified the major feedbacks and implemented the changes

Part1.png

We do not have a concrete conclusion from the easy level results, however, for the difficult level, we can derive from the test results that users generally take a shorter period of time to create a resume using Flauntpage as compared to using MS Words.

Another point to note is also as highlighted in the graph, the user actually took a much longer time when using FlauntPage. This was because the user is entirely new to FlauntPage and is not accustomed to the interactive features that FlauntPage provides such as drag & drop. He spent spent more time sorting the resume content into the required format as compared to when he was using MS Word.

The above findings allow us to realize the importance of some form of guidance or tutorial to be made available for 1st time users to let them get used to using some of the features in our application.


2. With the ratings provided by the testers in Part 2, team ploted a graph (time vs quality) to identify the correlation between MS Word and FlauntPage

The team expected a curve very much different from the one we plotted based on the results of our part 2 User Test. We could have missed out or overlooked certain points in conducting this test as we had expected quality of MS Word resumes to increase as time increases. & we had hope that the curve of FlauntPage would be near or catching up to the curve of MS Word as time increases.

However, according to the graph plotted above, we can see that for a shorter time taken to create a resume, the quality is much higher for FlauntPage as compared to MS Word. This is true because users require more time to build their resume using MS Word with regards to formatting. However, with FlauntPage, formatting of the resume is taken care of by the application and the end result, (when compared by testers), FlauntPage generated resumes which are of higher quality as compared to MS Word.

We also observed that most testers rated the resumes based on the following factors:

  • Resumes with lines, colors or bold headings
  • Consistency and neatness
Part2.png

Improvements

Improvements

Instant Resume Template (Finalized Layout):

Resume1.jpg Resume2.jpg


Cover Letter Template (Finalized Layout):

Coverletter1.jpg