ATD: Collaborative Research: Extremal Dependence and Change-Point Detection Methods for High-Dimensional Data Streams with Applications to Network Cybersecurity

ATD：协作研究：高维数据流的极端依赖性和变点检测方法及其在网络网络安全中的应用

基本信息

批准号：
1830175
负责人：
George Michailidis
金额：
$ 8.8万
依托单位：
University of Florida
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-08-01 至 2021-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1830175&HistoricalAwards=false
关键词：
ATD Collaborative Research Extremal Dependence

项目摘要

The project is motivated by the need to develop advanced network monitoring tools coupled with automated statistical methods for the quick detection of Internet traffic anomalies due to ongoing attacks or impending cybersecurity threats. Emphasis is placed on detecting cybersecurity threats such as highly distributed malware infections, which can launch coordinated and crippling distributed denial of service attacks on the nation's Internet infrastructure. This will be achieved through a study of the so-called darknet traffic data. Malicious actors in the network systematically probe the Internet space for vulnerable or misconfigured devices. In doing so, they automatically send data to the entire Internet address space, which includes the space of unused Internet addresses. This destined-to-nowhere traffic is indicative of malware infection attempts or stealthy vulnerability scanning. The investigators aim to develop and deploy specialized tools that allow cyber-security analysts to efficiently analyze darknet traffic data. The research involves a team of computer engineers and statisticians, who will work closely together to implement a prototype system for detecting as well as mapping and identifying world-wide malicious activity in the Internet. The project will create and communicate to the public a set of simple-to-interpret risk indices that summarize the current darknet threat activity. This effort will potentially enable the prevention and mitigation of cybersecurity network traffic threats.Understanding Internet threats, which continue to evolve due to the dynamic nature of Internet actors and the rapid expansion of the Internet of Things ecosystem, requires adequate data at fine-grained spatial and temporal scales. The project team has access to unique cyber-security data collected at Merit Network, Inc. that capture Internet-wide activity including network scanning, malware propagation, denial of service attacks, and network outages. This data consists of unsolicited Internet traffic destined to a routed but unused Internet address space, referred to as a darknet. This project will develop algorithmic and software infrastructure to collect and organize darknet data into high-dimensional, multivariate data streams, and will study statistical methods based on (i) extremal dependence, (ii) change-point detection, and/or (iii) high-dimensional sparse signal detection and recovery to inform the construction of Internet threat indices that quantify the risk of malicious scanning, degree of network vulnerability, risk of denial of service attacks, etc. Statistics of extremes in high-dimensional setting is a challenging problem since it requires the modeling/estimation of an infinite-dimensional parameter---the spectral measure. Using multivariate regular variation, this project will study novel hyper-graphical models that quantify and provide interpretable abstractions for the simultaneous occurrence of extremes in high-dimensions. Using limit theory for maxima of dependent variables, the project team will address open theoretical problems on the characterization of extremal dependence hyper-graphs and sparse signal detection in high-dimension. This analysis will lead to the development of novel threat indices that exhibit spatial dependence that will be analyzed with fast, scalable change-point detection algorithms. The new change-point methodology is designed to achieve large computational gains vis-a-vis standard approaches without compromising statistical accuracy and would be a significant contribution to the analysis of large data streams.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的动机是需要开发先进的网络监控工具以及自动统计方法，以快速检测由于持续攻击或迫在眉睫的网络安全威胁而导致的互联网流量异常。重点是检测网络安全威胁，例如高度分布的恶意软件感染，这些威胁可以对国家互联网基础设施发起协调一致的、造成严重破坏的分布式拒绝服务攻击。这将通过对所谓的暗网流量数据的研究来实现。网络中的恶意行为者系统地探测互联网空间中是否存在易受攻击或配置错误的设备。在此过程中，它们会自动将数据发送到整个互联网地址空间，其中包括未使用的互联网地址空间。这种注定无处可去的流量表明恶意软件感染尝试或秘密漏洞扫描。调查人员的目标是开发和部署专用工具，使网络安全分析师能够有效分析暗网流量数据。这项研究涉及一个由计算机工程师和统计学家组成的团队，他们将密切合作，实施一个原型系统，用于检测、映射和识别全球范围内的互联网恶意活动。该项目将创建并向公众传达一组易于解释的风险指数，总结当前的暗网威胁活动。这项工作将有可能有助于预防和缓解网络安全网络流量威胁。由于互联网参与者的动态性质和物联网生态系统的快速扩张，互联网威胁不断演变，因此需要了解足够的细粒度空间数据和时间尺度。项目团队可以访问 Merit Network, Inc. 收集的独特网络安全数据，这些数据捕获互联网范围内的活动，包括网络扫描、恶意软件传播、拒绝服务攻击和网络中断。该数据包含发送至已路由但未使用的互联网地址空间（称为暗网）的未经请求的互联网流量。该项目将开发算法和软件基础设施，以收集暗网数据并将其组织成高维、多变量数据流，并将研究基于（i）极值依赖性、（ii）变化点检测和/或（iii）的统计方法高维稀疏信号检测和恢复，为构建互联网威胁指数提供信息，量化恶意扫描的风险、网络脆弱性程度、拒绝服务攻击的风险等。高维环境下的极端情况统计是一个具有挑战性的问题因为它需要无限维参数的建模/估计——谱测量。该项目将使用多元规则变化来研究新颖的超图形模型，该模型可以量化高维极端情况的同时发生并提供可解释的抽象。利用因变量最大值的极限理论，项目团队将解决极值依赖超图表征和高维稀疏信号检测的开放理论问题。这种分析将导致新型威胁指数的开发，这些威胁指数表现出空间依赖性，并将通过快速、可扩展的变化点检测算法进行分析。新的变点方法旨在在不影响统计准确性的情况下实现相对于标准方法的巨大计算增益，并将对大数据流的分析做出重大贡献。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。

项目成果

期刊论文数量（9）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

A Statistical Framework for Detecting Electricity Theft Activities in Smart Grid Distribution Networks

用于检测智能电网配电网络中窃电活动的统计框架

DOI：
10.1109/jsac.2019.2952181
发表时间：
2020-01-01
期刊：
IEEE Journal on Selected Areas in Communications
影响因子：
16.4
作者：
Jin Tao;G. Michailidis
通讯作者：
G. Michailidis

Multiple Change Points Detection in Low Rank and Sparse High Dimensional Vector Autoregressive Models

低秩稀疏高维向量自回归模型中的多变化点检测

DOI：
10.1109/tsp.2020.2993145
发表时间：
2020-01
期刊：
IEEE Transactions on Signal Processing
影响因子：
5.4
作者：
Bai, Peiliang;Safikhani, Abolfazl;Michailidis, George
通讯作者：
Michailidis, George

Sequential change-point detection in high-dimensional Gaussian graphical models

高维高斯图模型中的顺序变化点检测

DOI：
发表时间：
2020-06
期刊：
Journal of machine learning research
影响因子：
6
作者：
Keshavarz, Hossein;Michailidis, George;Atchade, Yves
通讯作者：
Atchade, Yves

The interdisciplinary nature of network monitoring: Advantages and disadvantages

网络监控的跨学科性质：优点和缺点

DOI：
10.1080/08982112.2021.1974034
发表时间：
2021-10
期刊：
Quality Engineering
影响因子：
2
作者：
Stevens, Nathaniel T.;Wilson, James D.;Driscoll, Anne R.;McCulloh, Ian;Michailidis, George;Paris, Cecile;Parker, Peter;Paynabar, Kamran;Perry, Marcus B.;Reisi;et al
通讯作者：
et al

Change Point Estimation in a Dynamic Stochastic Block Model

动态随机块模型中的变化点估计