III: Small: Collaborative Research: A novel paradigm for detecting complex anomalous patterns in multi-modal, heterogeneous, and high-dimensional multi-source data sets

III：小型：协作研究：一种检测多模态、异构和高维多源数据集中复杂异常模式的新范式

• 批准号: 1954409
• 负责人: Feng Chen
• 金额: $ 20.59万
• 依托单位: University of Texas at Dallas
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1954409&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; novel

One of the greatest challenges in modern data analysis is to identify subtle, complex anomalous patterns (subsets of a data set that are novel or unexpected) within ubiquitous multi-modal, heterogeneous, and high-dimensional multi-source data sets in the current big data era. The detection of such salient patterns is an indispensable tool for knowledge mining and discovery in important applications across many fields of science, engineering, and business, including the early detection of infectious disease outbreaks, crime hotspots, network intrusions, false advertising, cyber botnets, customer activity monitoring and user profiling, and fraudulent medical claims, among others. The project research goal is to develop a new and innovative paradigm for discovering complex and subtle anomalous patterns in ubiquitous multi-modal, heterogeneous, and high-dimensional multi-source datasets in the current big data era. The key idea is to generalize the idea of meta-analysis from the statistical community and to reframe the problem as a search over all subsets of nonparametric statistical tests that are conducted on individual record-level features, in order to find the subsets (anomalous patterns) that are jointly significant. The project is focused on real-world problems related to biosurveillance and cybersecurity with two challenging applications: early detection of rare and infectious disease outbreaks (e.g., foodborne, Hantavirus, yellow fever) and Sybil attacks (e.g., spammers, fake users, and compromised normal users). The integrated education plan includes the development of new courses offered at the Master of Science program in Computer Science and Informatics and outreach to underrepresented groups. The outcomes of this project will be widely disseminated to broader audience via tutorials and workshops.The research objectives of this project are: (1) the development of nonparametric tests for modeling anomalous information of multi-source datasets; (2) learning heterogeneous dependencies among nonparametric tests; (3) detecting anomalous patterns from an extremely large set of nonparametric tests; and (4) making the detected anomalous patterns interpretable in the context of multi-modal, heterogeneous, and high-dimensional data. The research approach includes the development of (1) nonparametric tests on individual record level features that provide consistent representations of anomalous information from multiple heterogeneous input modalities, such as image, text, video, and multiple sensor streams; (2) deep structured and adversarial methods capable of learning robust hierarchical dependency structures of nonparametric tests using unlabeled training data; (3) fast, scalable combinatorial optimization methods capable of accurately detecting salient anomalous patterns from billion-size nonparametric tests; and (4) transparent and interpretable methods capable of explaining the predicted anomalous patterns by identifying training instances and features that are most responsible for the predictions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代数据分析中最大的挑战之一是在当前大数据时代无处不在的多模式，异质和高维多源数据集中确定微妙的，复杂的异常模式（新颖或意外的数据集的子集（新颖或意外的数据集）。检测这种显着模式是在许多科学，工程和业务领域的重要应用中知识挖掘和发现的必不可少的工具，包括早期发现传染病爆发，犯罪点，网络入侵，虚假广告，网络僵尸网络，网络僵尸网络，客户活动监测以及用户分析以及欺诈性的医学要求等。项目研究目标是开发一种新的创新范式，以发现无处不在的多模式，异质和高维多源数据集中的复杂而微妙的异常模式。关键思想是概括从统计界的荟萃分析的概念，并将问题重新构架为对所有记录级特征进行的非参数统计测试的搜索，以便找到共同重要的子集（异常模式）。该项目的重点是与生物监测和网络安全有关的现实问题，并具有两个具有挑战性的应用：早期发现罕见和传染病暴发（例如，食源性疾病，hantavirus，hantavirus，黄热病）和Sybil攻击（例如，垃圾邮件发送者，垃圾邮件发送者，假用户和正常使用者）。综合教育计划包括开发计算机科学和信息学硕士课程提供的新课程，并向代表性不足的群体开发。该项目的结果将通过教程和讲习班广泛传播给更广泛的受众。该项目的研究目标是：（1）开发非参数测试，用于建模多源数据集的异常信息； （2）在非参数测试中学习异质依赖性； （3）从一组极大的非参数测试中检测异常模式； （4）使检测到的异常模式在多模式，异质和高维数据的背景下进行解释。研究方法包括开发（1）对单个记录水平特征的非参数测试，这些特征提供了来自多种异构输入模式的异常信息的一致表示，例如图像，文本，视频和多个传感器流； （2）使用未标记的训练数据学习非参数测试的鲁棒层次依赖性结构的深层结构和对抗方法； （3）能够准确地检测出数十亿个非参数测试的快速，可扩展组合优化方法； （4）能够通过识别最负责预测的培训实例和特征来解释预测异常模式的透明和可解释的方法。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响审查标准通过评估来进行评估的。

项目成果

A Primal-Dual Subgradient Approach for Fair Meta Learning

• DOI: 10.1109/icdm50108.2020.00091
• 发表时间: 2020-11
• 期刊: 2020 IEEE International Conference on Data Mining (ICDM)
• 作者: Chengli Zhao;Feng Chen;Zhuoyi Wang;L. Khan

Block-Structured Optimization for Anomalous Pattern Detection in Interdependent Networks

• DOI: 10.1109/icdm.2019.00137
• 发表时间: 2019-11
• 期刊: 2019 IEEE International Conference on Data Mining (ICDM)
• 作者: Fei Jie;Chunpai Wang;Feng Chen;Lei Li;Xindong Wu

Distributionally Robust Optimization for Deep Kernel Multiple Instance Learning

• 发表时间: 2021
• 作者: Hitesh Sapkota;Yiming Ying;F. Chen;Qi Yu

Calibrated Nonparametric Scan Statistics for Anomalous Pattern Detection in Graphs

用于图形中异常模式检测的校准非参数扫描统计

• 发表时间: 2022
• 期刊: Thirty-Sixth AAAI Conference on Artificial Intelligence
• 作者: Chunpai Wang, Daniel Neill

Fairness-Aware Online Meta-learning

公平意识在线元学习

• 发表时间: 2021
• 期刊: In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
• 作者: Zhao, Chen;Chen, Feng;Thuraisingham, Bhavani
• 通讯作者: Thuraisingham, Bhavani