HeaderSIS.jpg

Altitude User Testing

From IS480
Jump to navigation Jump to search
Altitude logo black.jpg
User Testing
Overview Project Management Documentation

Schedule

Scope


Summary of User Tests Documentation
User Test Details Methodology Documents for User Test User Test Results
1 Date: 24 Sep 2013(Tues)
Participants: 3
Location: IMC Singapore
Web Application UT 1 Briefing PowerPoint Slides

UT 1 Execution Plan
UT Instructions document Pre-Test Survey Post-Test Survey

UT 1 Results
2 Date: 13 Nov 2013(Wed)
Participants: 6
Location: IMC Singapore
Web Application

Timer method

UT 2 Pre-test briefing PowerPoint Slides

UT 2 Briefing Slides
UT 2 Execution Plan
UT 2 Instructions document (IMC Staff)
UT 2 Instructions document (Germany)
Post-Test Survey

UT2 Results

Compatible with the following internet brower:
FireFox | Download it here
Google Chrome | Download it here

User Testing 2

Objectives

The main objectives of this user test are:

  • To verify that our SIS web application has the capability to switch ontologies between different industries.
  • To test the functionalities of the web application in the deployed environment
  • To verify the quality of our application as compared to UT 1


UT 2’s hypothesis

  • Our application is easy to use.
    • On average, each user can able to complete each section of the RFP process within the allocated time frame
  • We are better aligned to the need to our clients.
    • The average usability ratings of UT2 will be higher than what that of UT1.


Details

Pre-briefing

  • Participants : 5 IMC staff (IMC Singapore and Germany users)
  • Venue : IMC Singapore
  • Date : 12th Nov 2013
  • Duration : 30 mins (Online call)

UT 2

  • Participants: 2 IMC staff
  • Venue : IMC Singapore
  • Date: 13th Nov 2013
  • Duration : 3 hours (Each user takes 1 hour to complete)


Method

  • The web application was deployed in SAP HANA and was tested using test user's desktop. Each user was given the UT instruction sheet to fill up their feedback.


Data Collection

  • Collection of qualitative metrics
    • We observe how the users interact with our web application
    • At the end of UT, users complete the post-test survey with their feedback


Features Tested

No. Feature Name Features Description
1 Login Allow the user to login to our system
2 Take in Requirements from RFP Allow the user to input requirements to the system, and for the system to ask the right questions
3 Recommend Options The system will rank and recommend the top 3 qualified options
4 Modify Recommended Options Allows the user to configure product option to better align to requirements
5 Review and Edit Proposal The system will display the proposal in an editable state
6 Download PDF Allows the user to download proposal in PDF
User Test 2 Results


Observation

UT observation picture.png


Feedback

UT feedback picture.png


Comparison Between User Testing 1 and User Testing 2

-Altitude- Expected Timing.png


UT 1 vs UT 2.png


  • Based on the UT 2 results, it shows that our SIS web application is getting better in terms of its process flow and UI design.
  • Functions-wise the application is a new attempt in solving complex guided selling business problem using the use of ontology concepts and IMC reasoning platform
  • Technology-wise, there are components such as SAPUI5, HANA Cloud, SPRING, OWL and IMC reasoning platform that make this application not only technically complex but also academically an interesting project to learn and acquire solid enterprise level application development experience.


Most improvement features


User Testing 1 vs User Testing 2

User Testing 1 :: Old CSS Theme SAPUI5 v1.141 User Testing 2 :: New CSS Theme SAPUI5 v1.143
ThemeBefore.png ThemeAfter.png
User Testing 1 :: Create New Proposal [Professionally aligned Fields & Date Validation] User Testing 2 :: Create New Proposal [ Quick Creation of Proposal & View]
1aftUT.png 1afterUT2.png
User Testing 1 :: Gathering Requirement [ Busy Indicator ] User Testing 2 :: Gathering Requirement [ Busy Indicator, Delete Qns & Re-answer ]
2aftUT.png After UT2 gathering requirements.png
User Testing 1 :: Review & Edit Proposal [ Height of RichTextEditor Increased ] User Testing 2 :: Review & Edit Proposal [ Dynamic Proposal Content Insert ]
3aftUT.png
3afterUT2.png
User Testing 1 :: Creation of PDF [ Truncated texts using JSPDF ] User Testing 2 :: Creation of PDF [ New Structure using iText ]
OldPDF.png
NewPDF.png




Photo Gallery



Video Gallery


UT 2 interview

Ut2 interview.png
Click here to hear what our client feels about our web application


User Test 1


How User Testing was Conducted

Objectives

The aim of this user testing (UT) is to

  • Validate the team’s approach of taking in requirements via Q&A
  • Validate UI heuristics
  • Ensure tested functions are working


Purpose

  • Validate the application’s UI heuristics
  • Validate our approach of taking in requirements via Q&A procedure
    • Test if the system functions are working
    • Ask the user on the satisfactory level of these functions
      • E.g. If ≥ 50% “Yes”, the function is validated
  • Test the team’s hypotheses:
    • It will not take more than 5 seconds to find out how to answer the question
    • Non-experienced sales person will find this beneficial
      • Benchmark: ≥ 75% of non-server-SMEs will accurately
      • Benchmark: ≥ 100% of respondents strongly agree “this will be useful in other industries”.
    • The questioning have to be easy enough to be understood without domain knowledge
    • Benchmark: ≥ 75% of non-Server-SMEs never misunderstood our
      • Questions and Answers. “Never misunderstood” means “Agree to large extent” in our questionnaire.
    • Salespersons will generally prefer the team questioning methodology over the As-Is process (highlighting + writing down requirements) in extracting requirements from an RFP.
      • Benchmark: ≥ 75% say “Yes”.


Details

Part 1

  • Participants: 3 IMC staffs
  • Venue : IMC Singapore
  • Date: 26th Sep 2013
  • Duration : 2 hours

Part 2

  • Participants: 1 Germany user
  • Venue: IMC Singapore
  • Mode of Communication : Online call
  • Date: 26th Sep 2013
  • Duration : 1 hour 30 minutes


Method

  • The web application was deployed in SAP HANA and was tested using test user's desktop. Each user was given the UT instruction sheet to fill up the UT results.


Data Collection

  • Collecting of qualitative metrics
    • We observe how the users when they are using our web application
    • At the start of UT, users have to complete the pre-test survey
    • At the end of UT, users have to complete the post-test survey


Features Tested

No. Feature Name Features Description
1 Login Allow the user to login to our system
2 Take in Requirements from RFP Allow the user to input requirements to the system, and for the system to ask the right questions
3 Recommend Options The system will rank and recommend the top 3 qualified options
4 Modify Recommended Options Allows the user to configure product option to better align to requirements
5 Review and Edit Proposal The system will display the proposal in an editable state
6 Download PDF Allows the user to download proposal in PDF
User Test Results


Observation(T1).png Feedbacks(T1).png


UT Improvement

Before VS After UT

Create New Proposal

During UT After UT
1b4UT.png
1aftUT.png


Gathering Requirement

During UT After UT
2b4UT.png
2aftUT.png

Review and Edit Proposal

During UT After UT
3b4UT.png
3aftUT.png



Photo Gallery



Video Gallery

UT 1 Process

UTresult.png
Please click here to view our UT 1 Video


UT 1 interview

UTinterview.jpg
Click here to hear what our client feels about our web application