Difference between revisions of "IS480 Team wiki: 2012T1 One-hit Wonder Final Wiki Usability Study"
Line 198: | Line 198: | ||
The team expected a curve very much different from the one we plotted based on the results of our part 2 User Test. We could have missed out or overlooked certain points in conducting this test as we had expected quality of MS Word resumes to increase as time increases. & we had hope that the curve of FlauntPage would be near or catching up to the curve of MS Word as time increases. | The team expected a curve very much different from the one we plotted based on the results of our part 2 User Test. We could have missed out or overlooked certain points in conducting this test as we had expected quality of MS Word resumes to increase as time increases. & we had hope that the curve of FlauntPage would be near or catching up to the curve of MS Word as time increases. | ||
+ | |||
+ | [[Image:Part2.png|center]] | ||
However, according to the graph plotted above, we can see that for a shorter time taken to create a resume, the quality is much higher for FlauntPage as compared to MS Word. This is true because users require more time to build their resume using MS Word with regards to formatting. However, with FlauntPage, formatting of the resume is taken care of by the application and the end result, (when compared by testers), FlauntPage generated resumes which are of higher quality as compared to MS Word. | However, according to the graph plotted above, we can see that for a shorter time taken to create a resume, the quality is much higher for FlauntPage as compared to MS Word. This is true because users require more time to build their resume using MS Word with regards to formatting. However, with FlauntPage, formatting of the resume is taken care of by the application and the end result, (when compared by testers), FlauntPage generated resumes which are of higher quality as compared to MS Word. | ||
Line 204: | Line 206: | ||
* Resumes with lines, colors or bold headings | * Resumes with lines, colors or bold headings | ||
* Consistency and neatness | * Consistency and neatness | ||
− | |||
− | |||
</div> | </div> | ||
<div style="background: #7D9C9F; border:3px solid #7D9C9F; -webkit-box-shadow: 0px 0px 6px #7D9C9F; padding: 10px;"></div> | <div style="background: #7D9C9F; border:3px solid #7D9C9F; -webkit-box-shadow: 0px 0px 6px #7D9C9F; padding: 10px;"></div> |
Revision as of 22:33, 5 December 2012
Contents
Usability Study: User Testing 2
Usability Study Recap
Usability Study Recap |
HEURISTIC EVALUATION
In order to find out the usability of Flauntpage, we conducted a Heuristic Evaluation.
This HE focuses mainly on 8 Usability Heuristics focus points as follows:
- Be consistent
- Focus on tasks
- Provide feedback
- Follow established conventions
- Speak the user’s language
- Prevent errors
- Minimize clicks
- Avoid scrolling
Refer to detailed analysis of Heuristic Evaluation here
USER TESTING 1
Our first User Testing was conducted to gather more in-depth insights on the usability of FlauntPage and to ensure that the deployed version is running smoothly.
We also gathered many valuable feedbacks on how we could improve the workflow due to varying human behavior and other user interface improvements
Refer to detailed analysis of UT1 here
Difference between UT1 & UT2
UT1 | UT2 | |
Objectives | To ensure that FlauntPage is performing well on the server with no major bugs and errors caught in the process. | To find out the effectiveness of FlauntPage and how convenient it is for users to create employment documents using our application as compared to other tools like Microsoft Word.
To find out the effectiveness and value of FlauntPage and how convenient it is for companies to find suitable candidates as compared to other channels (e.g Other job portals, newspaper, company website etc) |
Execution | Guided.
Users assigned a role and given detailed test cases with guided steps to follow. Users rated difficulty and gave comments for each task completed. |
Unguided.
Part I: Users assigned a role and given a resume template to create a replica of the resume using Microsoft words and FlauntPage. Timings will taken for creation of documents and we later observed users behavior when using FlauntPage. |
UT2 Objective
UT2 Objective |
Execution Period: 5th November 2012 - 7th November 2012 (3 days)
Job Seeker:
To find out the effectiveness of FlauntPage and how convenient it is for users to create employment documents using our application as compared to other tools like Microsoft Word.
Job Seeker | |
Total no. of testers | 16 |
Browser used | Google Chrome & Firefox |
Age range | 18 - 30 |
No. of testers assigned difficult level | 10 |
No. of testers assigned easy level | 6 |
Company:
To find out the effectiveness and value of FlauntPage and how convenient it is for companies to find suitable candidates as compared to other channels (e.g Other job portals, newspaper, company website etc)
Company | |
Total no. of testers | 2 |
Browser used | Google Chrome |
Company name/Tester | Richemont Luxury (Singapore) Pte Ltd and Hiring Consultant |
Procedure
Procedure |
Part 1:
- Tester will be assigned to complete a resume using either the easy or difficult version of the sample resume information
- Create a resume using Microsoft Word (Activity timed)
- Testers are required to format the layout and imagine that they are sending this resume out to apply for a job
- Login to dev.flauntpage.com and create a resume using the same piece of information provided (Activity timed)
- Team will record the time taken in completing the resume for both the MS Word and Flauntpage version.
- Navigate around the application – Flauntpage freely to try out other functionalities
- Answer a few questions on their general feelings of our application – FlauntPage
Download sample of easy template here
Download sample of difficult template here
Download sample of user overall experience feedback here
Part 2:
1. With all the resumes collected from the previous testers, we will get another group of testers to measure the quality of the resume. (Total 9)
- 3 Testers were given 6 sets of EASY resumes (consist of 3 EASY MS Word & 3 EASY FlauntPage version, randomized)
- 3 Testers were given 10 sets of DIFFICULT resumes (consist of 5 DIFFICULT MS Word & 5 DIFFICULT FlauntPage version, randomized)
- 3 Testers were given 16 sets of EASY & DIFFICULT resumes (consist of 3 EASY MS Word & 3 EASY FlauntPage, 5 DIFFICULT MS Word & 5 DIFFICULT FlauntPage version, randomized)
2. Testers were then tasked to rate each resume between the scale of 1 to 5 (1 being the lowest)
- They were asked to find their base ratings out of the pile first in order to ensure that subsequent ratings will be consistent
Results & Analysis
Results & Analysis |
1. Based on the feedbacks provided by the testers in Part 1 of the testing, team identified the major feedbacks and implemented the changes
We do not have a concrete conclusion from the easy level results, however, for the difficult level, we can derive from the test results that users generally take a shorter period of time to create a resume using Flauntpage as compared to using MS Words.
Another point to note is also as highlighted in the graph, the user actually took a much longer time when using FlauntPage. This was because the user is entirely new to FlauntPage and is not accustomed to the interactive features that FlauntPage provides such as drag & drop. He spent spent more time sorting the resume content into the required format as compared to when he was using MS Word.
The above findings allow us to realize the importance of some form of guidance or tutorial to be made available for 1st time users to let them get used to using some of the features in our application.
2. With the ratings provided by the testers in Part 2, team ploted a graph (time vs quality) to identify the correlation between MS Word and FlauntPage
The team expected a curve very much different from the one we plotted based on the results of our part 2 User Test. We could have missed out or overlooked certain points in conducting this test as we had expected quality of MS Word resumes to increase as time increases. & we had hope that the curve of FlauntPage would be near or catching up to the curve of MS Word as time increases.
However, according to the graph plotted above, we can see that for a shorter time taken to create a resume, the quality is much higher for FlauntPage as compared to MS Word. This is true because users require more time to build their resume using MS Word with regards to formatting. However, with FlauntPage, formatting of the resume is taken care of by the application and the end result, (when compared by testers), FlauntPage generated resumes which are of higher quality as compared to MS Word.
We also observed that most testers rated the resumes based on the following factors:
- Resumes with lines, colors or bold headings
- Consistency and neatness
Improvements
Improvements |