ATD: Collaborative Research: Extremal Dependence and Change-Point Detection Methods for High-Dimensional Data Streams with Applications to Network Cybersecurity

ATD：协作研究：高维数据流的极端依赖性和变点检测方法及其在网络网络安全中的应用

• 批准号: 1830175
• 负责人: George Michailidis
• 金额: $ 8.8万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1830175&HistoricalAwards=false
• 关键词: ATD; Collaborative; Research; Extremal; Dependence

The project is motivated by the need to develop advanced network monitoring tools coupled with automated statistical methods for the quick detection of Internet traffic anomalies due to ongoing attacks or impending cybersecurity threats. Emphasis is placed on detecting cybersecurity threats such as highly distributed malware infections, which can launch coordinated and crippling distributed denial of service attacks on the nation's Internet infrastructure. This will be achieved through a study of the so-called darknet traffic data. Malicious actors in the network systematically probe the Internet space for vulnerable or misconfigured devices. In doing so, they automatically send data to the entire Internet address space, which includes the space of unused Internet addresses. This destined-to-nowhere traffic is indicative of malware infection attempts or stealthy vulnerability scanning. The investigators aim to develop and deploy specialized tools that allow cyber-security analysts to efficiently analyze darknet traffic data. The research involves a team of computer engineers and statisticians, who will work closely together to implement a prototype system for detecting as well as mapping and identifying world-wide malicious activity in the Internet. The project will create and communicate to the public a set of simple-to-interpret risk indices that summarize the current darknet threat activity. This effort will potentially enable the prevention and mitigation of cybersecurity network traffic threats.Understanding Internet threats, which continue to evolve due to the dynamic nature of Internet actors and the rapid expansion of the Internet of Things ecosystem, requires adequate data at fine-grained spatial and temporal scales. The project team has access to unique cyber-security data collected at Merit Network, Inc. that capture Internet-wide activity including network scanning, malware propagation, denial of service attacks, and network outages. This data consists of unsolicited Internet traffic destined to a routed but unused Internet address space, referred to as a darknet. This project will develop algorithmic and software infrastructure to collect and organize darknet data into high-dimensional, multivariate data streams, and will study statistical methods based on (i) extremal dependence, (ii) change-point detection, and/or (iii) high-dimensional sparse signal detection and recovery to inform the construction of Internet threat indices that quantify the risk of malicious scanning, degree of network vulnerability, risk of denial of service attacks, etc. Statistics of extremes in high-dimensional setting is a challenging problem since it requires the modeling/estimation of an infinite-dimensional parameter---the spectral measure. Using multivariate regular variation, this project will study novel hyper-graphical models that quantify and provide interpretable abstractions for the simultaneous occurrence of extremes in high-dimensions. Using limit theory for maxima of dependent variables, the project team will address open theoretical problems on the characterization of extremal dependence hyper-graphs and sparse signal detection in high-dimension. This analysis will lead to the development of novel threat indices that exhibit spatial dependence that will be analyzed with fast, scalable change-point detection algorithms. The new change-point methodology is designed to achieve large computational gains vis-a-vis standard approaches without compromising statistical accuracy and would be a significant contribution to the analysis of large data streams.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的激励是由于需要开发高级网络监控工具以及自动统计方法，以快速检测由于持续攻击或即将发生的网络安全威胁而引起的互联网流量异常。重点是检测网络安全威胁，例如高度分布的恶意软件感染，这些威胁可能会引发对国家互联网基础架构的协调和残酷的分布拒绝服务攻击。 这将通过对所谓的暗网流量数据进行研究来实现。网络中的恶意参与者系统地探索了脆弱或错误配置的设备的Internet空间。在此过程中，他们会自动将数据发送到整个Internet地址空间，其中包括未使用的Internet地址的空间。这一注定的到处流量表明了恶意软件感染的尝试或隐身脆弱性扫描。 研究人员旨在开发和部署专业工具，以使网络安全分析师有效地分析DarkNet交通数据。这项研究涉及一个计算机工程师和统计学家团队，他们将紧密合作，以实施一个原型系统来检测以及在互联网中绘制和识别全球恶意活动。该项目将与公众建立并沟通一组简单截止的风险指数，以总结当前的DarkNet威胁活动。 这项努力有可能使网络安全网络交通威胁的预防和缓解。理解互联网威胁，由于互联网参与者的动态性质和物联网生态系统的快速扩展，这种威胁继续发展，需要在细粒度的空间上进行足够的数据和时间尺度。该项目团队可以访问在Merit Network，Inc。收集的独特网络安全数据，该数据捕获了网络范围的活动，包括网络扫描，恶意软件传播，拒绝服务攻击和网络中断。该数据由原定于路由但未使用的Internet地址空间（称为DarkNet）的未经请求的Internet流量组成。该项目将开发算法和软件基础架构，以将DarkNet数据收集和组织到高维的多元数据流中，并将基于（i）极端依赖性，（ii）变更点检测和/或（iii）研究统计方法。高维的稀疏信号检测和恢复，以告知量化恶意扫描风险，网络脆弱程度，拒绝服务攻击的风险等互联网威胁指标的构建。高维环境中极端的统计数据是一个具有挑战性的问题由于它需要对无限维参数的建模/估计 - 光谱度量。使用多元规则变化，该项目将研究新型的超图模型，这些模型可以量化并为高度极端发生的同时出现可解释的抽象。使用限制理论，对于因变量的最大值，项目团队将解决有关极端依赖性超图表和高维度中稀疏信号检测的开放理论问题。该分析将导致新的威胁指数的发展，这些威胁指标表现出空间依赖性，这些依赖性将通过快速，可扩展的更改点检测算法进行分析。新的更改点方法旨在实现相对于标准方法的大量计算收益，而不会损害统计准确性，这将是对大型数据流的分析的重要贡献。这项奖项反映了NSF的法定任务，并被认为值得一提通过基金会的智力优点和更广泛的影响评估标准通过评估来支持。

项目成果

Broader impacts of network monitoring: Its role in government, industry, technology, and beyond

• DOI: 10.1080/08982112.2021.1974036
• 发表时间: 2021-10
• 期刊: Quality Engineering
• 影响因子: 2
• 作者: Nathaniel T. Stevens;James D. Wilson;Anne R. Driscoll;I. Mcculloh;G. Michailidis;Cécile Paris;K. Paynabar;M. Perry;Mostafa Reisi-Gahrooei;Srijan Sengupta;Ross Sparks

Change Point Estimation in a Dynamic Stochastic Block Model

• 发表时间: 2018-12
• 期刊: J. Mach. Learn. Res.
• 作者: M. Bhattacharjee;M. Banerjee;G. Michailidis

The interdisciplinary nature of network monitoring: Advantages and disadvantages

网络监控的跨学科性质：优点和缺点

• DOI: 10.1080/08982112.2021.1974034
• 发表时间: 2021
• 期刊: Quality Engineering
• 影响因子: 2
• 作者: Stevens, Nathaniel T.;Wilson, James D.;Driscoll, Anne R.;McCulloh, Ian;Michailidis, George;Paris, Cecile;Parker, Peter;Paynabar, Kamran;Perry, Marcus B.;Reisi-Gahrooei, Mostafa
• 通讯作者: Reisi-Gahrooei, Mostafa

Detection of False Data Injection Attacks in Smart Grids Based on Forecasts

• DOI: 10.1109/smartgridcomm.2018.8587473
• 发表时间: 2018-10
• 期刊: 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm)
• 作者: M. Kallitsis;Shrijita Bhattacharya;G. Michailidis

Foundations of network monitoring: Definitions and applications

• DOI: 10.1080/08982112.2021.1974033
• 发表时间: 2021-10
• 期刊: Quality Engineering
• 影响因子: 2
• 作者: Nathaniel T. Stevens;James D. Wilson;Anne R. Driscoll;I. Mcculloh;G. Michailidis;Cécile Paris;K. Paynabar;M. Perry;Mostafa Reisi-Gahrooei;Srijan Sengupta;Ross Sparks