ISSS608 2017-18 T3 Assign Lu Yanzhang Data Preparation Methodology

VAST Challenge 2018 MC3:
Who hurts the brid?

INTRODUCTION

DATA PREPARATION & METHODOLOGY

OBSERVATION AND INSIGHTS

Back to Dropbox

Tools

The following tools have been used in this assignment

1. Python - Timestamp transformation and new data source generation.

The following packages are used in this assignment: pandas, numpy, glob, datetime.

2. JMP Pro - Data preparation

3. Tableau - Visualization

4. Gephi - Social network modeling and visualization

Timestamp Transformation in Python

The raw timestamp format is the second record from '2015-05-11 14:00:00'.

For further use of timestamp data, the format needs to be transformed to YYYY/MM//DD rather than raw second format.

Join operation among diverse tables in JMP

Join the tables where the source or target is “suspicious” and select out the suspicious transactions for the further visualizations in Tableau and social network analytics in Gephi

Social network modeling in Gephi

Import the suspicious data file into Gephi and model the data with two methodologies:

1. Eigenvalue centrality for vertex importance calculation.

2. Modularity for clustering calculation.

Visualization in Tableau

1. Visualize the communication table by day and by month to interpret the growth from 2015 to 2017.

2. Visualize the suspicious staffs' activities.