2013-14 Term 1 G2 TeamYOLO A9 Plan

From Interaction Design and Prototyping
Jump to: navigation, search
Home Assignments G1 G2 Technology



Experiment Objectives & Plan

Objective

  • During our user tests, some of our testers felt that the setting up of diner's initials was tedious and unnecessary. We would like to find out if that's true with a A/B Split Test Experiment.
  • Hence, for this web experiment, we would like to determine if omitting the process of naming each generated face would speed up the progress of splitting a bill unevenly.
    • At the same time, we also wish to test if omitting the naming process would cause users to be prone to making more mistakes during the bill splitting process.


Summarised Plan

  1. Scope: Setting up of diner’s initials.
    • Some of our testers felt it was tedious and unnecessary.
  2. Purpose: To determine if omitting the process of naming each generated face will speed up the process of splitting a bill unevenly. At the same time, we also wish to test if this will cause users to be prone to making more mistakes during the bill splitting process.
  3. Task: Split the given complex bill among six diners using BillSplitter’s Split Unevenly Feature, on two different layouts.
  4. Independent variable(s): Layout and Flow of Application (A/B Split Test)
    • Layout #1: Enter Initials before proceeding (Current BillSplitter Process)
    • Layout #2: Omitted naming of initials process. Diners would be distinguished with different colored faces.
  5. Dependent variable:
    • Total Time taken to Split a Bill for each layout
    • Number of diners with the wrong allocation of the bill

Acceptance Criteria for Experiment

  • For Dependent Variable 1 - Total Time taken:
    • Minimum accepted score: 1 minute.
    • Maximum accepted score: 6 minutes.
  • For Dependent Variable 2 - Number of "diners" with the wrong allocation of the bill
    • Minimum accepted score: 0 diners incorrectly allocated.
    • Maximum accepted score: 6 diners incorrectly allocated.

A/B Test Experiment Flow (Overview)

TeamYOLO-A9-ExperimentFlow.png

Overview of Flow

Counter Balancing

TeamYOLO-A9-CounterBalancing.png

Justification for the Above Setup

One common issue with within-subject test designs, is that they are subjected to carryover effects, in particular, the practice carryover effect.

This effect occurs when testers get much better over time because of practice (from previous tests); thereby potentially causing inaccuracy in our data collection (i.e. total time taken). For example, testers that went through Layout #1 may complete Layout #2 much quicker due to the "practice" they received from Layout #1.

Hence, we counterbalanced the order of our layouts, to overcome this practice carryover effect.

Source: http://psych.csufresno.edu/price/psych144/counterbalancing.html

Layouts Screenshots

TeamYOLO-A9-Layout1.png TeamYOLO-A9-Layout2.png
  • Layout #1: Enter Initials before proceeding (Current BillSplitter Process)
  • Layout #2: Omitted naming of initials process. Diners would be distinguished with different colored faces.

Capturing of Dependent Variables

Dependent Variable Methods of Collection
Total Time Taken to Split a Bill Each layout would have a hidden embedded timer, that triggers the moment our tester lands on the site.

Upon submission of the bill inputs, the timer stops clocking, and the time value (in seconds) will be stored in our database for further analysis.

Number of Incorrect Dish Allocation Upon form submission of the bill inputs, validation would be performed on each diner's bill input. Correct and incorrect answers would be marked as 'T' or ‘F’ respectively, and will be stored in our database for further analysis.



Database Structure

TeamYOLO Dependent Variable Data Structure.png

Possible Sample Data

TeamYOLO Dependent Variable Possible Sample Data.png

All these collected data would then, be populated on a graph to better visualise our findings.

Capturing of Successful Splits across Experimental Links

TeamYOLO-A9-SuccessfulSplits.png
In order to carry out a slightly more balanced analysis, we would need to have the same amount of testers/submission for each experimental link. (e.g. 15 successful splits for each experimental link).

This would allow us to acquire a fairly more accurate reading on our dependent variable (i.e. the average of the total time taken to split the bill across two different experimental link).

At this point of writing, the above snapshot are merely sample datas.

Note: Capturing of successful splits are done via pixel firing on our "thank-you" page.


Analytical Tools

Team-YOLO-Analytics.png