HeaderSIS.jpg

IS480 Team wiki: 2015T1 A3XY TestDocuments

From IS480
Jump to navigation Jump to search
A3xy team logo1.png



A3xy HomeIcon.png

HOME

  A3xy UserIcon.png

ABOUT US

  A3xy Project.png

PROJECT OVERVIEW

  A3xy StatsIcon.png

PROJECT MANAGEMENT

  A3xy PapersIcon.png

DOCUMENTATION

 
Technical Documents Design Documents Test Documents WP SubPlane.png Meeting Minutes Project Documents Deployment


No. Plan
1 Regression Test Plan


User Test 1


Objective : Gather feedback on the usability of the portal

The tasks assigned can be found here: A3XY User Test 1 Plan
Demographic

Users Students
Number of Users 12
Venue SMU school campus


Brief information on the participants:

  • 12 SMU students
  • Previous module classmates
  • 8 Male and 4 Female
  • 3 teams of 4 students each
  • Each team: 1 Importer, 1 Exporter, 1 Advising Bank, 1 Issuing Bank


Goals:

  • Test LC application, LC issuance, LC advise, LC authorization and acknowledgement
  • Test the ease of use of the current version of the web application
  • Test the understanding of the process concept after platform usage


Feedback Collected:

  • Likes
  1. The user interface seems to be neat
  2. The process is pretty straight-forward
  3. The colours are nice
  4. Easy to follow the step by step process on IBM software
  5. Cool to receive SMS
  • Dislikes
  1. The form is too big
  2. The login process is complicated
  3. The home page icons are too big
  4. It is hard to understand how to use the software at first go
  5. The validations are irritating



Key Findings:

  • One group took more time to complete the while process than the other two.
  • Group members using IBM BPM software required some assistance while using it.
  • Based on comments: Change the images used for Importer/Exporter buttons on home page.
  • Based on comments: Provide an edit option for the forms before submitting.
  • Based on observations: Separate the functioning of maker and checker in bank roles to make them work independently.


User Test 2


Objective : Are we building an understandable portal?

The tasks assigned can be found here: A3XY User Test 2 Plan
Demographic

Users Students
Number of Users 5
Venue SMU school campus


Brief information on the participants:

  • Experienced with background in Consumer Internet Banking
  • Understand banking terms
  • Have not being exposed to the software being used (IBM BPM)


Goals:

  • Students should find the BPM portal and Trade Finance portal user friendly.
  • Students should like the web interface of the portal.
  • Students should understand the process of LC Application and Document Presentation.


Feedback Collected: The feedback collected can be found here: A3XY User Test 2 Results

Results:

  • Time Spent in finishing the LC Application process (in mins)
A3xy UT2 result1.png











  • Time Spent in finishing the Document Presentation process (in mins)
A3xy UT2 result2.png











  • Did you understand the full process?
A3xy UT2 result3.png












  • Was the application easy to use?
A3xy UT2 result4.png












Key Findings:

  • Participants find the technical names confusing sometimes. They have to go back to the Trade Finance home page every time in order to recall the meaning of different terms.
  • Participants pointed out that it would be better if the URL of the document is clickable on the portal itself rather than copy pasting
  • Participants find the IBM BPM portal easy to use and straightforward with minimum clicks involved.
  • Participants liked the user interface of the Trade Finance portal.
  • 80% of the participants understood the process of LC Application and Document Presentation using this application.
  • 60% of the participants did not find the application easy to use.
  • Participants were a bit confused about the ISO and SWIFT messaging standards display.


User Test 3


Objective : Process Verification

The tasks assigned can be found here: A3XY User Test 3 Plan
Demographic

Users Professors
Number of Users 2
Venue SIS MR 4.3


Brief information on the participants:

  • Experienced with background in Trade Finance
  • Work at Ngee Ann Polytechnic


Goals:

  • To gather suggestions and feedback after using the application.
  • To verify content and check consistency throughout the platform.
  • To find pain points identified by the Professors while using the platform and trade portal.
  • To find out whether the platform would act as a good teaching tool or not.


Usability:
Ratings of various elements of the portal

A3xy UT3 result.png














User Test 4


Objective : Compare the usability of the platform between first-time and old users.

The tasks assigned can be found here: A3XY User Test 4 Plan

Demographic

Users Students
Number of Users 12 (6 old users, 6 new users)
Venue SMU school campus


Brief information on the participants:
Group A (old users)

  • Have prior experience of using the platform
  • Have worked on the LC process
  • Have basic knowledge of Trade Finance from prior usage


Group B (new users)

  • Experienced with background in Consumer Internet Banking
  • Understand banking terms
  • Have not being exposed to the software being used (IBM BPM)



Results:

  • Time Spent in finishing the LC Application process (in mins)
A3xy UT4 result1.png











  • Time Spent in finishing the Document Presentation process (in mins)
A3xy UT4 result2.png











  • Average Time Spent in finishing the Shipping Guarantee, Trust Receipt and Banker’s guarantee processes (in mins)
A3xy UT4 result3.png











  • Did you understand the full process?
A3xy UT4 result4.png












  • Was the application easy to use?
A3xy UT4 result5.png












  • Results for quizzes for both the groups. (Average number of answers answered correctly)
A3xy UT4 result6.png










Key Findings:

  • Group A participants feel that the new interface is a big improvement from the previous version.
  • Group A participants feel that the new version provides a more comprehensive flow of the Trade Finance process and its components.
  • Group A participants have on average 25% better performance compared to their performance in a previous user test.
  • Group A participants have on average 15% better performance compared to the performance of Group B on average.
  • All participants found the glossary to be really useful for referencing to the definitions.
  • Group B participants do not face the issues that Group A faced in the previous user tests and on average have a 10% better performance for the same processes.
  • 12.5% of Group B participants felt that the constant movement between the portal and IBM BPM was confusing
  • All participants find individual components of the application easy to use.


User Test 5

Objective : Heuristic Evaluation of the built application.

The tasks assigned can be found here: A3XY User Test 5 Plan

Demographic

Users Students
Number of Users 8
Venue SOE CR 3.4


Brief information on the participants:

  • Have basic knowledge of Trade Finance from prior study


Goals:

  • Students should be able to navigate around the website easily
  • Students should understand the requirements for various input fields without additional help
  • Students should find the application visually appealing


Results:

  • Rating based on the student’s experience while navigating the website where 1 is hard and 5 is easy
A3xy UT5 result1.png










  • Rating based on clarity of requirements for input fields
A3xy UT5 result2.png










  • Rating based on visual appeal of the website
A3xy UT5 result3.png










Key Findings:

  • In general, participants find it easy to navigate around that website. However, some participants still take some time to understand the jump from trade portal to trade operations
  • Participants are able to understand the requirement for all input fields easily.
  • Participants find the trade portal visually appealing but there is still room for improvement.
  • Some participants feel a requirement for a feature that show a summary of their lc transactions.