IS480 Team wiki: 2015T1 A3XY TestDocuments
Technical Documents | Design Documents | Test Documents | Meeting Minutes | Project Documents | Deployment |
|
Users | Students |
Number of Users | 12 |
Venue | SMU school campus |
Brief information on the participants:
- 12 SMU students
- Previous module classmates
- 8 Male and 4 Female
- 3 teams of 4 students each
- Each team: 1 Importer, 1 Exporter, 1 Advising Bank, 1 Issuing Bank
Goals:
- Test LC application, LC issuance, LC advise, LC authorization and acknowledgement
- Test the ease of use of the current version of the web application
- Test the understanding of the process concept after platform usage
Feedback Collected:
- Likes
- The user interface seems to be neat
- The process is pretty straight-forward
- The colours are nice
- Easy to follow the step by step process on IBM software
- Cool to receive SMS
- Dislikes
- The form is too big
- The login process is complicated
- The home page icons are too big
- It is hard to understand how to use the software at first go
- The validations are irritating
Key Findings:
- One group took more time to complete the while process than the other two.
- Group members using IBM BPM software required some assistance while using it.
- Based on comments: Change the images used for Importer/Exporter buttons on home page.
- Based on comments: Provide an edit option for the forms before submitting.
- Based on observations: Separate the functioning of maker and checker in bank roles to make them work independently.
User Test 2
Objective : Are we building an understandable portal?
The tasks assigned can be found here: A3XY User Test 2 Plan
Demographic
Users | Students |
Number of Users | 5 |
Venue | SMU school campus |
Brief information on the participants:
- Experienced with background in Consumer Internet Banking
- Understand banking terms
- Have not being exposed to the software being used (IBM BPM)
Goals:
- Students should find the BPM portal and Trade Finance portal user friendly.
- Students should like the web interface of the portal.
- Students should understand the process of LC Application and Document Presentation.
Feedback Collected:
The feedback collected can be found here: A3XY User Test 2 Results
Results:
- Time Spent in finishing the LC Application process (in mins)
- Time Spent in finishing the Document Presentation process (in mins)
- Did you understand the full process?
- Was the application easy to use?
Key Findings:
- Participants find the technical names confusing sometimes. They have to go back to the Trade Finance home page every time in order to recall the meaning of different terms.
- Participants pointed out that it would be better if the URL of the document is clickable on the portal itself rather than copy pasting
- Participants find the IBM BPM portal easy to use and straightforward with minimum clicks involved.
- Participants liked the user interface of the Trade Finance portal.
- 80% of the participants understood the process of LC Application and Document Presentation using this application.
- 60% of the participants did not find the application easy to use.
- Participants were a bit confused about the ISO and SWIFT messaging standards display.
User Test 3
Objective : Process Verification
The tasks assigned can be found here: A3XY User Test 3 Plan
Demographic
Users | Professors |
Number of Users | 2 |
Venue | SIS MR 4.3 |
Brief information on the participants:
- Experienced with background in Trade Finance
- Work at Ngee Ann Polytechnic
Goals:
- To gather suggestions and feedback after using the application.
- To verify content and check consistency throughout the platform.
- To find pain points identified by the Professors while using the platform and trade portal.
- To find out whether the platform would act as a good teaching tool or not.
Usability:
Ratings of various elements of the portal
User Test 4
Objective : Compare the usability of the platform between first-time and old users.
The tasks assigned can be found here: A3XY User Test 4 Plan
Demographic
Users | Students |
Number of Users | 12 (6 old users, 6 new users) |
Venue | SMU school campus |
Brief information on the participants:
Group A (old users)
- Have prior experience of using the platform
- Have worked on the LC process
- Have basic knowledge of Trade Finance from prior usage
Group B (new users)
- Experienced with background in Consumer Internet Banking
- Understand banking terms
- Have not being exposed to the software being used (IBM BPM)
Results:
- Time Spent in finishing the LC Application process (in mins)
- Time Spent in finishing the Document Presentation process (in mins)
- Average Time Spent in finishing the Shipping Guarantee, Trust Receipt and Banker’s guarantee processes (in mins)
- Did you understand the full process?
- Was the application easy to use?
- Results for quizzes for both the groups. (Average number of answers answered correctly)
Key Findings:
- Group A participants feel that the new interface is a big improvement from the previous version.
- Group A participants feel that the new version provides a more comprehensive flow of the Trade Finance process and its components.
- Group A participants have on average 25% better performance compared to their performance in a previous user test.
- Group A participants have on average 15% better performance compared to the performance of Group B on average.
- All participants found the glossary to be really useful for referencing to the definitions.
- Group B participants do not face the issues that Group A faced in the previous user tests and on average have a 10% better performance for the same processes.
- 12.5% of Group B participants felt that the constant movement between the portal and IBM BPM was confusing
- All participants find individual components of the application easy to use.
User Test 5
Objective : Heuristic Evaluation of the built application.
The tasks assigned can be found here: A3XY User Test 5 Plan
Demographic
Users | Students |
Number of Users | 8 |
Venue | SOE CR 3.4 |
Brief information on the participants:
- Have basic knowledge of Trade Finance from prior study
Goals:
- Students should be able to navigate around the website easily
- Students should understand the requirements for various input fields without additional help
- Students should find the application visually appealing
Results:
- Rating based on the student’s experience while navigating the website where 1 is hard and 5 is easy
- Rating based on clarity of requirements for input fields
- Rating based on visual appeal of the website
Key Findings:
- In general, participants find it easy to navigate around that website. However, some participants still take some time to understand the jump from trade portal to trade operations
- Participants are able to understand the requirement for all input fields easily.
- Participants find the trade portal visually appealing but there is still room for improvement.
- Some participants feel a requirement for a feature that show a summary of their lc transactions.