Difference between revisions of "1718t1is428T12"
(Created page with "== Problem Statement ==") |
|||
Line 1: | Line 1: | ||
− | == Problem Statement == | + | [[File:team12logo.jpg|200px|frameless|center]] |
+ | <p></p><br/> | ||
+ | <!--Header--> | ||
+ | {|style="background-color:#F5F5F5; color:#4d79ff; padding: 10 0 10 0;" width="100%" cellspacing="0" cellpadding="0" valign="top" border="0" | | ||
+ | | style="padding:0.2em; font-size:100%; background-color:#F3BE32; text-align:center; color:#1D1D1D" width="10%" | | ||
+ | [[1718t1is428T12 |<font color="#1D1D1D" size=2 face="Gotham"><b>PROPOSAL</b></font>]] | ||
+ | |||
+ | | style="background:none;" width="1%" | | ||
+ | | style="padding:0.2em; font-size:100%; background-color:#F5F5F5; border-bottom:0px solid #3D9DD7; text-align:center; color:#1D1D1D" width="10%" | | ||
+ | [[1718t1is428T12_Poster|<font color="#1D1D1D" size=2 face="Gotham"><b>POSTER</b></font>]] | ||
+ | |||
+ | | style="background:none;" width="1%" | | ||
+ | | style="padding:0.2em; font-size:100%; background-color:#F5F5F5; border-bottom:0px solid #3D9DD7; text-align:center; color:#1D1D1D" width="10%" | | ||
+ | [[1718t1is428T12_Application|<font color="#1D1D1D" size=2 face="Gotham"><b>APPLICATION</b></font>]] | ||
+ | |||
+ | | style="background:none;" width="1%" | | ||
+ | | style="padding:0.2em; font-size:100%; background-color:#F5F5F5; border-bottom:0px solid #3D9DD7; text-align:center; color:#1D1D1D" width="10%" | | ||
+ | [[1718t1is428T12_Research Paper|<font color="#1D1D1D" size=2 face="Gotham"><b>RESEARCH PAPER</b></font>]] | ||
+ | |} | ||
+ | <!--/Header--> | ||
+ | |||
+ | <!--Content--> | ||
+ | <br/> | ||
+ | |||
+ | ==<div style="background: #F3BE32; padding: 15px; line-height: 0.3em; text-indent: 15px; font-size:18px; font-family:Gotham"><font color= #1D1D1D>Problem Statement</font></div>== | ||
+ | <div style="font-family:Gotham;font-size:16px"> | ||
+ | In data analytics and visualization, there are unique challenges faced in the security realm as compared to conventional Business Intelligence analytics; rare events (i.e. attacks) within massive volumes of data need to be detected. Organizations commonly use Security Information and Event Management software solutions to gather security events from various systems and logs, by deploying real-time alert tools optimized to do event-driven signature matching for a subset of the overall gathered data. As attacks are more sophisticated these days, threats are more difficult to detect with signature-based systems. | ||
+ | |||
+ | The aim of the project is to help organizations identify high-level security threats and attacks by analyzing large amounts of security data (i.e. event logs) we have collected, such that security analysts are able to deal with such threats and attacks with the findings without being required to have strong data science proficiency. | ||
+ | |||
+ | |||
+ | ==<div style="background: #F3BE32; padding: 15px; line-height: 0.3em; text-indent: 15px; font-size:18px; font-family:Gotham"><font color= #1D1D1D>Background Survey of Related Works</font></div>== | ||
+ | {| class="wikitable" style="background-color:#FFFFFF;" width="100%" | ||
+ | |- | ||
+ | ! style="font-weight: bold;background: #fffee4;color:#1D1D1D;width: 50%;" | Visualizations | ||
+ | ! style="font-weight: bold;background: #fffee4;color:#1D1D1D;" | Explaination | ||
+ | |- | ||
+ | | [[Image:team12_viz1.jpg|550px|center]] | ||
+ | <br> | ||
+ | <center>Data source: http://www.digitalattackmap.com/ </center> | ||
+ | || | ||
+ | Digital Attack Map is a live data visualization of DDoS attacks around the globe, built through a collaboration between Google Ideas and Arbor Networks. The tool surfaces anonymous attack traffic data to let users explore historic trends and find reports of outages happening on a given day. | ||
+ | |- | ||
+ | | [[Image:team12_viz2.jpg|550px|center]] | ||
+ | <br> | ||
+ | <center>Data source: http://www.trendmicro.com.sg/sg/security-intelligence/current-threat-activity/global-botnet-map/index.html</center> | ||
+ | || | ||
+ | Global botnet threat activity map by Trend Micro shows the monitored malicious network activity to identify command-and-control servers. It enables security analyst to identify which computers are being attack how many often over the past 2 weeks. | ||
+ | |- | ||
+ | | [[Image:team12_viz3.jpg|550px|center]] | ||
+ | <br> | ||
+ | <center>Data source: https://www.akamai.com/us/en/solutions/intelligent-platform/visualizing-akamai/real-time-web-monitor.jsp </center> | ||
+ | || | ||
+ | Global Internet conditions map by Akamai. With this real-time data they identify the global regions target of the greatest web attack traffic, cities with the slowest web connections (latency) and geographic areas with the most web traffic (traffic density). | ||
+ | |} | ||
+ | |||
+ | ==<div style="background: #F3BE32; padding: 15px; line-height: 0.3em; text-indent: 15px; font-size:18px; font-family:Gotham"><font color= #1D1D1D>Tools / Libraries</font></div>== | ||
+ | <div style="font-family:Gotham;font-size:16px"> | ||
+ | *Tableau | ||
+ | *Github | ||
+ | *Microsoft Excel | ||
+ | *Adobe Creative Suites | ||
+ | *Sketch | ||
+ | *Javascript: sigma.js, vis.js, three.js, D3.js | ||
+ | *Python | ||
+ | </div> | ||
+ | |||
+ | ==<div style="background: #F3BE32; padding: 15px; line-height: 0.3em; text-indent: 15px; font-size:18px; font-family:Gotham"><font color= #1D1D1D>Technical Challenges</font></div>== | ||
+ | {| class="wikitable" style="background-color:#FFFFFF;" width="100%" | ||
+ | |- | ||
+ | ! style="font-weight: bold;background: #fffee4;color:#1D1D1D; width: 50%;" | Key Technical Challenges | ||
+ | ! style="font-weight: bold;background: #fffee4;color:#1D1D1D;" | Detailed Description | ||
+ | ! style="font-weight: bold;background: #fffee4;color:#1D1D1D;" | Proposed Solution | ||
+ | |- | ||
+ | | <center> Unfamiliar with D3.js libraries </center> || | ||
+ | D3.js is a JavaScript library for producing dynamic, interactive data visualizations in web browsers. | ||
+ | || | ||
+ | *Go for the d3 workshop | ||
+ | *Self learning | ||
+ | *Peer Learning | ||
+ | |- | ||
+ | | <center> Data Cleaning and Transformation </center> || | ||
+ | The data set are in text format and many other different format. Integration are challenging as there are a lot of manual work to be done. | ||
+ | || | ||
+ | * Delegate workload for cleaning datasets | ||
+ | |- | ||
+ | | <center> Data Unavailability </center> || | ||
+ | Many actual security attacks and defense data are paid version. | ||
+ | || | ||
+ | *Research databases using school links through the library portal | ||
+ | |- | ||
+ | | <center> Determining the Most Optimal Interactive Elements </center> || | ||
+ | In order to enable users to understand the data sets, interactive elements needs to be suitable for this project | ||
+ | || | ||
+ | *Develop storyboard | ||
+ | *Research on security visualization | ||
+ | |} | ||
+ | |||
+ | ==<div style="background: #F3BE32; padding: 15px; line-height: 0.3em; text-indent: 15px; font-size:18px; font-family:Gotham"><font color= #1D1D1D>Roles & Milestones </font></div>== | ||
+ | |||
+ | ===Project Timeline=== | ||
+ | [[Image: projecttimeline_team12.png |800px|center]] | ||
+ | |||
+ | ==<div style="background: #F3BE32; padding: 15px; line-height: 0.3em; text-indent: 15px; font-size:18px; font-family:Gotham"><font color= #1D1D1D>References</font></div>== | ||
+ | *Databases: http://vizsec.org/data/ | ||
+ | *Botnet Dataset: https://www.uvic.ca/engineering/ece/isot/datasets/ | ||
+ | *D3.js: https://d3js.org/ | ||
+ | *Detect virus: https://www.virustotal.com | ||
+ | |||
+ | ===Comments=== | ||
+ | Please leave comments here. | ||
+ | |||
+ | <!--/Content--> |
Revision as of 19:21, 16 October 2017
Contents
Problem Statement
In data analytics and visualization, there are unique challenges faced in the security realm as compared to conventional Business Intelligence analytics; rare events (i.e. attacks) within massive volumes of data need to be detected. Organizations commonly use Security Information and Event Management software solutions to gather security events from various systems and logs, by deploying real-time alert tools optimized to do event-driven signature matching for a subset of the overall gathered data. As attacks are more sophisticated these days, threats are more difficult to detect with signature-based systems.
The aim of the project is to help organizations identify high-level security threats and attacks by analyzing large amounts of security data (i.e. event logs) we have collected, such that security analysts are able to deal with such threats and attacks with the findings without being required to have strong data science proficiency.
Background Survey of Related Works
Visualizations | Explaination |
---|---|
|
Digital Attack Map is a live data visualization of DDoS attacks around the globe, built through a collaboration between Google Ideas and Arbor Networks. The tool surfaces anonymous attack traffic data to let users explore historic trends and find reports of outages happening on a given day. |
|
Global botnet threat activity map by Trend Micro shows the monitored malicious network activity to identify command-and-control servers. It enables security analyst to identify which computers are being attack how many often over the past 2 weeks. |
|
Global Internet conditions map by Akamai. With this real-time data they identify the global regions target of the greatest web attack traffic, cities with the slowest web connections (latency) and geographic areas with the most web traffic (traffic density). |
Tools / Libraries
- Tableau
- Github
- Microsoft Excel
- Adobe Creative Suites
- Sketch
- Javascript: sigma.js, vis.js, three.js, D3.js
- Python
Technical Challenges
Key Technical Challenges | Detailed Description | Proposed Solution |
---|---|---|
D3.js is a JavaScript library for producing dynamic, interactive data visualizations in web browsers. |
| |
The data set are in text format and many other different format. Integration are challenging as there are a lot of manual work to be done. |
| |
Many actual security attacks and defense data are paid version. |
| |
In order to enable users to understand the data sets, interactive elements needs to be suitable for this project |
|
Roles & Milestones
Project Timeline
References
- Databases: http://vizsec.org/data/
- Botnet Dataset: https://www.uvic.ca/engineering/ece/isot/datasets/
- D3.js: https://d3js.org/
- Detect virus: https://www.virustotal.com
Comments
Please leave comments here.