Difference between revisions of "IS480 Team wiki: 2013T1 Kungfu Panda X-Factor"
Yh.koon.2010 (talk | contribs) |
|||
(One intermediate revision by the same user not shown) | |||
Line 169: | Line 169: | ||
____________________________________________________________________________________________________________________________ | ____________________________________________________________________________________________________________________________ | ||
+ | |||
+ | === Credit Engine Calibration === | ||
+ | As we were unable to obtain sample loan data, upon consultation with the OCBC head of credit modeling, we were advised to do what 'new' banks do(small banks which are just starting up). | ||
+ | [[Image:Calibration1.png|584x439px]] | ||
+ | |||
+ | 1. Calibrate the weightages and thresholds of each rule through pairwise comparison. | ||
+ | * This allows us to input the relative importance of each rule. | ||
+ | |||
+ | 2. Normalize the pairwise comparison to a score of 800 (Based on FICO's framework). | ||
+ | * We normalized the results from the pairwise comparison using Eigen Vectors calculation. | ||
+ | [[Image:Calibration2.png|584x439px]] | ||
+ | |||
+ | 3. Calibrate the Accept and Reject Thresholds through simulation. | ||
+ | * Generated a sample of 1000 loan data profiles with a 3.8% default rate (double of Singapore Bank's default rate). | ||
+ | * Define accept and reject thresholds to simulate a typical Singapore bank's Default Rate. (1.85% on average) | ||
+ | * Our calibration achieved a default rate between 1%-1.6%, typical to Singapore Banks | ||
+ | |||
+ | [[Image:Calibration3.png|584x439px]] | ||
===Mockup of Credit Approval Form=== | ===Mockup of Credit Approval Form=== |
Latest revision as of 14:23, 25 November 2013
Contents
- 1 X-Factor Component
- 2 Usability
- 3 Credit Engine Technology Evaluation
X-Factor Component
Usability
The User Interface of our SMUtBank Teller Application has been made to be as cutting-edge and as Native-like as possible. Much of which is absent from typical Websites.
Utilizing a couple of important features:
- Single-Page Design (Back and Forward arrows are still usable)
- Front-end validation for every field
- Useful Visual Effects
- High performance
- Page caching
- Minimal Clicks
- Simple to use
- Tight and fixed layout
Another important feature of User-Interface is a strong automated testing
User Interface
Single Page Design
We make use of a Single Page Design for a very strong reason: Speed. Many of the page "changes" rely on Backbone.js to swap elements in and out and manage data through the use of AJAX (Asynchronous Javascript and XML). The advantages of this are clear:
- All resources only needed to be loaded once
- Page requests are minimized to AJAX requests
- Better user experience and no page "whiteouts" (White page when loading), making it closer to the performance of a native application.
However, there are also some disadvantages to this method:
- Increased complexity in managing the web application, much more Javascript to manage the AJAX requests
- Much more logic checks in place.
Front-end validation
Every field and button click is validated and checked. After the field has been edited by the User, it will be checked immediately via Javascript to ensure basic requirements have been met, such as if it is an Integer, a Float, or a Varchar. There are some additional checks based on the functionality of the page, such as LTV ratio or if the account has a sufficient balance. Certain aspects of validation have to be handled by the backend service, such as checking for duplicate Identification numbers.
Validation is placed directly on the input field itself, through the use of HTML5 data attributes e.g. data-type="varchar". These data attributes allow the Javascript to view through the element and determine what validation process to apply to it.
Useful Visual effects
Almost every action, successful or not will produce a notification message 10 (15 for errors) seconds long and manually cleared. This is to give the user active feedback on his/her actions, it obstructs the screen slightly and forces the user to actively click it to ensure his/her acknowledgement.
As part of every important validation process, Signposts are extremely clear as to what the user should do. The input field is coloured red, or there will be a red notification message on the top right to signify a negative event. Should that not be enough, the borders of the screen will glow a slight red hue to clearly warn that the user has made a mistake. This idea itself was copied from modern First-Person shooter games who use reddish screens to warn of impending danger. It is applied to our application in great effect, without producing a conspicuous and inhibiting alert message as do most web applications.
Use of highlights and automated-screen scrolling is important too. Upon clicking certain unique elements, the user will be automatically scrolled to the point we want them to focus on. A good example is the Transaction History page, where a user is able to click a point on the graph and it will direct the user to a readable text version of the transaction. Errors that are out of sight will be scrolled to to ensure that the user sees the first error of the page, so as not give bring confusion as to why there is an error but not being able to see one (When it is actually out of view).
High Performance
Much of the Web application has been optimized for speed, without affecting accuracy. For example in generating the Individual Profiles for Credit Approval Simulation, all the information is stored in the Drop Down List itself, every option within the Drop Down List has its own data and displays itself when the user selects it. This is contrary to storing it in an Array and iterating through it everytime a user selects an ID from the Drop Down List that matches an ID in an array. Though however minor in performance impact, it helps keep the slickness of the web application feel constant through out.
The large usage of AJAX calls to facilitate communication with the Bank's core is also a key aspect of the web application. Every request and response is handled through AJAX only, this absolves of the user from fetch so many files from the server as well as vice-versa. Saving bandwidth, especially when 30-40 students are hammering away at the application at once. Which has been tested in a classroom environment to be steady throughout the session.
Page Caching
Due to the way the web application serves users, being a single page application, and never actually requiring a page refresh; All Pages are loaded during the initialization of the application and stored in the user's cache, should the user "change" a page, the page will be fetched templating-style and loaded onto the current Document Object Model, if necessary, an AJAX call will be made to the server to add data to it.
Minimal Clicks
The way controls are built allows the user to stay of the mouse if possible. Every single inch of the web application is usable without a mouse, that includes clicking on tabs or divs. Users are encouraged to minimize the rate of exchange between mouse and keyboard due to the nature of filling up forms in most typical situations. There are also clear indications of
Simple to use
The web application was designed for students in mind, and as well as how we could make the application as simple and straightforward as possible to use. The idea is simple, fill the form, and confirm it. In-between that part is the most important, ensuring that Users know where and what they have to key in and the banking jargon used, as handling any exceptions or errors that would otherwise throw most users off.
Some of the more subtle things:
- No need to aim for small checkboxes
- No pesky alert messages that forces users to click
- Back/Forward buttons are usable despite being a Single Page application
- Graphs are used to give the user a better idea of the situation
- Instructions are in tough places for first-time users
- Colour scheme is meant to be pleasing to the eye
- Buttons catch the User's eye to guide them to the important parts of the page
- Default/Clear buttons make life easier should there be mistakes
Tight and fixed layout
The layout is unique in regards to how websites are being designed. This web application does not have any responsive CSS and utilizes a fixed 1024 x 768 as an optimal resolution. However, the main reason is this, a fixed layout when used often will stay with the user. Should the screen change and the position of the input fields and buttons change, it could well put the user off, causing the user to search again for the fields he needs to type into.
Testing
Credit Engine Technology Evaluation
We evaluated three mainstream commercial decision engines to use as the base of our credit engine.
As Jess did not have a frontend rule repository and FICO was too costly, we finally settled on Drools as it satisfied our main criterias listed in the table below.
Credit Approval
The Credit scoring engine is a complex decision service that performs automatic credit evaluation and approval for mass consumer products such as Home Loans, Auto Loans, and Education Loans.
Credit scoring Process Flow
Credit approval process diagram
Credit Decision Engine Proof of Concept
POC for Credit Decision Engine
Market Research of Credit Engines
FICO Website: FICO Scoring Overview
FICO is a leading company in predictive analytics, specialising in scoring individual's credit-worthiness over a scale of 800 points. Our group used FICO's scoring model as our main reference when developing the rules for our Credit Engine.
Examples
____________________________________________________________________________________________________________________________
Credit Engine Calibration
As we were unable to obtain sample loan data, upon consultation with the OCBC head of credit modeling, we were advised to do what 'new' banks do(small banks which are just starting up).
1. Calibrate the weightages and thresholds of each rule through pairwise comparison.
- This allows us to input the relative importance of each rule.
2. Normalize the pairwise comparison to a score of 800 (Based on FICO's framework).
- We normalized the results from the pairwise comparison using Eigen Vectors calculation.
3. Calibrate the Accept and Reject Thresholds through simulation.
- Generated a sample of 1000 loan data profiles with a 3.8% default rate (double of Singapore Bank's default rate).
- Define accept and reject thresholds to simulate a typical Singapore bank's Default Rate. (1.85% on average)
- Our calibration achieved a default rate between 1%-1.6%, typical to Singapore Banks
Mockup of Credit Approval Form
User Interface of Credit Approval Form
Teaching Tool
The teaching tool allows users to simulate varying customer profiles (credit capacity) and also different weightages and thresholds for each of the credit rules. This gives students a hands on learning experience on generic credit scoring rules utilize by banks.
Teaching tool process flow
Demographic Information for Generating Teaching Tool Loan Profiles
We retrieved our demographic information from various sources. When presented with alternative sources, we chose the source which was more reliable (e.g. from an authority such as the Government).
The demographics are modeled after the Singapore population whenever possible.
Income:
- Median of $41760
- Typically ranges between $12, 240 to $300,000 a year
- Sources: http://stats.mom.gov.sg/iMAS_PdfLibrary/mrsd-msib2013.pdf, http://www.mom.gov.sg/Publications/mrsd_singapore_workforce_2012.pdf
Residence:
- Home ownership rate of 90.1%
- Source: http://stats.mom.gov.sg/iMAS_PdfLibrary/mrsd-msib2013.pdf
'Residence Stability:
- Average of 6 years (according to US survey as there was limited SG information)
- Source: http://www.creditsesame.com/blog/how-long-are-americans-staying-in-their-homes/
Job Stability:
- Median: 4.4 years (US survey)
- Source: http://www.forbes.com/sites/jeannemeister/2012/08/14/job-hopping-is-the-new-normal-for-millennials-three-ways-to-prevent-a-human-resource-nightmare/
No of Credit Cards
- Median of 3.3 cards per individual
- 75% of eligible cardholders have 2 or more cards
- Source: http://sg.finance.yahoo.com/news/singapore-top-asia-credit-cards-105414790.html
Credit and Banking History
- 1.85% derogatory records on average
- Source: http://www.btinvest.com.sg/system/assets/16730/Moody's%20-%20Banking%20System%20Outlook%20-%20Singapore%20071513.pdf
Loan Quantum (HDB Flat)
- Ranges from $320,000 to $820,000
- Average 3 room HDB, $355,444
- Average 4 room HDB, $490,352
- Average 5 room HDB, $563,427
- Source: http://www.hdb.gov.sg/fi10/fi10321p.nsf/w/BuyResaleFlatMedianResalePrices?OpenDocument
Age
- Source: http://www.singstat.gov.sg/statistics/browse_by_theme/population.html
- Normalized for average loan takers
- 25 - 54 years (74.1%)
- 55-64 years (14.4%)
- 65 years and > (11.5%)
Teaching tool Features (Input)
Data Generation
- Loan Profiles (Demographic information of loan applicants)
Scoring Engine Customization
- Rules customized to user’s preference
- Customizable Threshold/Ranges
- Customizable Weightage
Teaching tool Features (Output)
Breakdown
- # of approved/rejected/pending loans
- Min, Max, Average Credit Score
- Score-breakdown for individual loan profile