HeaderSIS.jpg

Difference between revisions of "IS480 Team wiki: 2015T2 Starteur User Testing 3"

From IS480
Jump to navigation Jump to search
(Created page with "<!--Logo--> center|300px <!--MAIN HEADER--> {|style="background-color:#FFFFFF; color:#FFFFFF; padding: 10 0 10 0;" width="100%" cellspacing="0" cellpad...")
 
m
 
(6 intermediate revisions by the same user not shown)
Line 57: Line 57:
  
 
==<div style="background: #354b60; padding: 15px; font-weight: bold; line-height: 0.3em; text-indent: 15px;letter-spacing:-0.08em;font-size:20px"><font color=#ffffff>Participants</font></div>==
 
==<div style="background: #354b60; padding: 15px; font-weight: bold; line-height: 0.3em; text-indent: 15px;letter-spacing:-0.08em;font-size:20px"><font color=#ffffff>Participants</font></div>==
* Number of educators: 3
+
* Number of educators: 4
* Number of students: 70
+
* Number of students: 80
 
* Roles of participants: Head-of-Department, Academic Director, Personal Development Trainer
 
* Roles of participants: Head-of-Department, Academic Director, Personal Development Trainer
  
  
 
<center>
 
<center>
[[File:IMG 1517.JPG|250px|link=]]
 
[[File:IMG 1518.JPG|250px|link=]]
 
 
</center>
 
</center>
  
 
==<div style="background: #354b60; padding: 15px; font-weight: bold; line-height: 0.3em; text-indent: 15px;letter-spacing:-0.08em;font-size:20px"><font color=#ffffff>Tasks</font></div>==
 
==<div style="background: #354b60; padding: 15px; font-weight: bold; line-height: 0.3em; text-indent: 15px;letter-spacing:-0.08em;font-size:20px"><font color=#ffffff>Tasks</font></div>==
* Each of the users was given a list of tasks to complete. Click [[Media:Task Instructions.docx| here]] to download tasklist.
+
* Each of the users was given a list of tasks to complete. Click [[Media:(UAT3)Task Instructions.docx| here]] to download tasklist.
 
* Before and after completing all the tasks, users are given a survey questionnaire to complete.
 
* Before and after completing all the tasks, users are given a survey questionnaire to complete.
  
 
==<div style="background: #354b60; padding: 15px; font-weight: bold; line-height: 0.3em; text-indent: 15px;letter-spacing:-0.08em;font-size:20px"><font color=#ffffff>Survey Results</font></div>==
 
==<div style="background: #354b60; padding: 15px; font-weight: bold; line-height: 0.3em; text-indent: 15px;letter-spacing:-0.08em;font-size:20px"><font color=#ffffff>Survey Results</font></div>==
 
===Collated Task Score===
 
===Collated Task Score===
 +
Educators were asked to rate the ease of completing the task ( 1 - Very Easy, 5 - Very Difficult)
 
<div style="text-align: left; width:99%;  margin-left:auto; margin-right:auto;">
 
<div style="text-align: left; width:99%;  margin-left:auto; margin-right:auto;">
 
{| class="wikitable" width="100%" font-family="Roboto Lt"
 
{| class="wikitable" width="100%" font-family="Roboto Lt"
Line 83: Line 82:
 
! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:20px; border:1px solid #999" | Overall Mean
 
! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:20px; border:1px solid #999" | Overall Mean
 
! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:30px; border:1px solid #999" | Remark
 
! style="font-family: Roboto Lt; text-align: center; bold;background: #34495e;color:#ecf0f1; width:30px; border:1px solid #999" | Remark
 
 
|-
 
|-
 
| style="text-align:center" | Task 1
 
| style="text-align:center" | Task 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 +
| style="text-align:center" | 2
 +
| style="text-align:center" | 2
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
| style="text-align:center" | 1
 
| style="text-align:center" | 3
 
 
| style="text-align:center" | 1.5
 
| style="text-align:center" | 1.5
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
Line 95: Line 93:
 
|-
 
|-
 
| style="text-align:center" | Task 2
 
| style="text-align:center" | Task 2
 +
| style="text-align:center" | 1
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
| style="text-align:center" | 4
+
| style="text-align:center" | 1.25
| style="text-align:center" | 2
 
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
  
Line 106: Line 104:
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
| style="text-align:center" | 2
+
| style="text-align:center" | 1
| style="text-align:center" | 4
+
| style="text-align:center" | 1
| style="text-align:center" | 2.25
+
| style="text-align:center" | 1.25
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
  
Line 114: Line 112:
 
| style="text-align:center" | Task 4
 
| style="text-align:center" | Task 4
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 +
| style="text-align:center" | 5
 +
| style="text-align:center" | 2
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
| style="text-align:center" | 1
+
| style="text-align:center" | 2.5
| style="text-align:center" | 1
 
| style="text-align:center" | 1.25
 
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
  
 
|-
 
|-
 
| style="text-align:center" | Task 5
 
| style="text-align:center" | Task 5
| style="text-align:center" | 2
 
| style="text-align:center" | 2
 
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
| style="text-align:center" | 4
+
| style="text-align:center" | 1
| style="text-align:center" | 2.25
+
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
 
 
|-
 
|-
 
| style="text-align:center" | Task 6
 
| style="text-align:center" | Task 6
 +
| style="text-align:center" | 2
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
| style="text-align:center" | 1
+
| style="text-align:center" | 1.5
| style="text-align:center" | 1.25
 
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
 
 
|-
 
|-
 
| style="text-align:center" | Task 7
 
| style="text-align:center" | Task 7
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
| style="text-align:center" | 4
 
| style="text-align:center" | 2
 
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
| style="text-align:center" | 2
+
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 
| style="text-align:center" | Nil
 
| style="text-align:center" | Nil
 
 
|-
 
|-
 
| style="text-align:center" | Task 8
 
| style="text-align:center" | Task 8
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1.75
 +
| style="text-align:center" | Nil
 +
|-
 +
| style="text-align:center" | Task 9
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1.25
 +
| style="text-align:center" |
 +
|-
 +
| style="text-align:center" | Task 10
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
 +
| style="text-align:center" | 1
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
 
| style="text-align:center" | 1
 
| style="text-align:center" | 1
| style="text-align:center" | 1.75
+
| style="text-align:center" | 1.5
| style="text-align:center" | Nil
+
| style="text-align:center" |  
 
+
|-
 +
| style="text-align:center" | Task 11
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" | 1
 +
| style="text-align:center" |
 
|-
 
|-
| style="text-align:center" | Task 9
+
| style="text-align:center" | Task 12
| style="text-align:center" | 3
+
| style="text-align:center" | 2
 
| style="text-align:center" | 3
 
| style="text-align:center" | 3
 +
| style="text-align:center" | 2
 +
| style="text-align:center" | 2
 +
| style="text-align:center" | 2.25
 +
| style="text-align:center" | h
 +
|-
 +
| style="text-align:center" | Task 13
 +
| style="text-align:center" | 1
 
| style="text-align:center" | 3
 
| style="text-align:center" | 3
 
| style="text-align:center" | 2
 
| style="text-align:center" | 2
| style="text-align:center" | 2.75
+
| style="text-align:center" | 1
| style="text-align:center" | Don't have to go to a new page to see which co-educator is attached to batch
+
| style="text-align:center" | 1.75
 +
| style="text-align:center" |  
 
|-
 
|-
 
</div>
 
</div>
  
[[File:UAT1 results.jpg|1000px|center|link=]]<br/>
+
[[File:UAT3 results.png|1000px|center|link=]]
  
  
Line 212: Line 238:
  
 
===User Study Summary===
 
===User Study Summary===
* Overall, users find the application easy to use, and the layout simple. intuitive to perform certain functions as it fits with other learning management systems that they have been using.  
+
* Overall, users find that the interface of the application has improved as compared to the previous version in UAT 1. It was a lot more intuitive.  
* In general, users are satisfied with most functions, except for a few
+
* Users are satisfied with most functions and find that functions were easy to use and suit their needs.
* User feedback for important features are fairly useful
+
* Users were satisfied with the report but feels that it needs more details/explanations.
 +
* Feedback from users were highly useful for report functions and fine-tuning functions.
  
 
===User Study Takeaways===
 
===User Study Takeaways===
* A consistency in usability for different users. Consider that different users have different capabilities in using such a system.  
+
* Important to keep solutions simple for users, be it a complex function or a relatively simple one.  
* With users' feedback, we identified areas of improvement on these functions. Eg. Add or Remove a co-educator
+
* With users' feedback, we managed to zero in on functions that needs more work.  
* Allow us to evaluate the usefulness of these features and future functions. Consideration on improvising the features and removing features that are redundant to users. Eg. Sort & Filter. Most feedback has been accounted for in our implementation plan, which will then be confirmed into subsequent development.
+
* The user study allowed us to identify areas for improvement and to evaluate how our primary & secondary functions can be further improved. Comments on individual report will be directed back to Starteur's development team and comments on batch report will be used to improve the usability of the batch report to an educator.

Latest revision as of 16:52, 5 April 2016

Starteur.jpg

HOME

ABOUT US

PROJECT OVERVIEW

PROJECT MANAGEMENT

DOCUMENTATION

Use Cases Technical Diagrams Design Documents Meeting Minutes User Testings Project Documents

Venue

  • Location: CHIJ St Nicholas Girls' School, Reactor ARC
  • Date: 14 - 23 Mar 2016
  • Time: -

Objectives

  • Verify that functionalities built are in line with user requirements
  • Determine if the user interface is intuitive
  • Gather feedback on current functionalities
  • Identify usability problems

Participants

  • Number of educators: 4
  • Number of students: 80
  • Roles of participants: Head-of-Department, Academic Director, Personal Development Trainer


Tasks

  • Each of the users was given a list of tasks to complete. Click here to download tasklist.
  • Before and after completing all the tasks, users are given a survey questionnaire to complete.

Survey Results

Collated Task Score

Educators were asked to rate the ease of completing the task ( 1 - Very Easy, 5 - Very Difficult)

UAT3 results.png


Task Participant 1 Participant 2 Participant 3 Participant 4 Overall Mean Remark
Task 1 1 2 2 1 1.5 Nil
Task 2 1 2 1 1 1.25 Nil
Task 3 1 2 1 1 1.25 Nil
Task 4 1 5 2 2 2.5 Nil
Task 5 1 1 1 1 1 Nil
Task 6 2 1 1 2 1.5 Nil
Task 7 1 1 1 1 1 Nil
Task 8 1 1 1 1 1.75 Nil
Task 9 1 1 2 1 1.25
Task 10 2 1 2 1 1.5
Task 11 1 1 1 1 1
Task 12 2 3 2 2 2.25 h
Task 13 1 3 2 1 1.75
# Issue Collated feedback
1 Text size Layout of the pages is simple and easy to see,
However, the text is quite faint and the font size can be bigger.
Bigger font size
Current layout is nice but text can be bigger.
Bigger font size.
2 Phrasing "Batch" is a little too definitive as teachers may only request a certain group of students to take the test. Consider changing to something less definitive and more generic.
3 Success Messages Application is simple to use and straightforward but I would like to know where I am at in terms of the steps in using the application.
I think it would be good if I can know whether some stuff I am doing is correct or not.
4 Lack of information hierarchy Everything is there on the page but I don't know where to look or what to focus on for certain stuff
I like the layout cause it is not confusing but it would be good if things are presented in a way that I can see certain important information.
5 Validation Do you have certain password restrictions?
Any password limits and email verification?
6 Any thing you will like to see in the application? A report for my cohort as well? It would be very insightful
It will be useful if I send the reminder myself. Because then I know how many days have passed since the reminder has been sent and send it again if necessary. I can control the frequency.
Allow me to send an email straight from this application.

Include a report for a cohort so that I can submit it for analysis with the other Year Heads.


User Study Summary

  • Overall, users find that the interface of the application has improved as compared to the previous version in UAT 1. It was a lot more intuitive.
  • Users are satisfied with most functions and find that functions were easy to use and suit their needs.
  • Users were satisfied with the report but feels that it needs more details/explanations.
  • Feedback from users were highly useful for report functions and fine-tuning functions.

User Study Takeaways

  • Important to keep solutions simple for users, be it a complex function or a relatively simple one.
  • With users' feedback, we managed to zero in on functions that needs more work.
  • The user study allowed us to identify areas for improvement and to evaluate how our primary & secondary functions can be further improved. Comments on individual report will be directed back to Starteur's development team and comments on batch report will be used to improve the usability of the batch report to an educator.