SaTC: CORE: Medium: Collaborative: Towards Robust Machine Learning Systems

SaTC：核心：媒介：协作：迈向稳健的机器学习系统

• 批准号: 1801751
• 负责人: Hao Chen
• 金额: $ 40万
• 依托单位: University of California-Davis
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1801751&HistoricalAwards=false
• 关键词: SaTC; CORE; Medium; Collaborative; Towards

Machine learning techniques, particularly deep neural networks, are increasingly integrated into safety and security-critical applications such as autonomous driving, precision health care, intrusion detection, malware detection, and spam filtering. A number of studies have shown that these models can be vulnerable to adversarial evasion attacks where the attacker makes small, carefully crafted changes to normal examples in order to trick the model into making incorrect decisions. This project's goal is to develop formal understandings of and defenses against these vulnerabilities through characterizing the relationship between adversarial and non-adversarial examples, developing mechanisms that exploit this relationship to support better detection of adversarial examples, and metrics and methods to demonstrate the robustness of machine learning models against them. Together, the theories, algorithms, and metrics developed will improve the robustness of machine learning systems, allowing them to be deployed more securely in mission-critical applications. The team will also make their datasets and source code publicly available and use them in their own courses and research with both graduate and undergraduate students, with particular efforts to include students from underrepresented groups in Science, Technology, Engineering and Math. The work will also support high school outreach programs and summer camps to attract younger students to study machine learning, security, and computer science.The project is organized around three main thrusts that combine to provide a holistic approach to modeling and defending against evasion attacks. The first thrust aims to characterize both normal and adversarial examples via systematic measurement studies. This includes considering different types of regions around specific examples (e.g., metric ball, manifold, and transformation-induced regions) and then characterizing the examples' vulnerability based on a number of algorithms for combining classifications of other examples in the nearby regions. The second thrust focuses on designing robust defenses against adversarial examples by using representative data points in a region, aggregating multiple data points, and using a diverse set of classifiers to reduce the vulnerability induced by using single data points or algorithms. The third thrust involves defining metrics for modeling robustness along with theories and algorithms that leverage those metrics to analyze model robustness. These include lower bounds of adversarial perturbation in metric balls, robustness metrics based on computational costs, analyses of the representativeness of new datasets relative to training data, and methods for leveraging human estimation of adversarialness.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习技术，尤其是深层神经网络，越来越多地集成到安全和安全的应用中，例如自动驾驶，精密医疗保健，入侵检测，恶意软件检测和垃圾邮件过滤。 许多研究表明，这些模型可能容易受到对抗性逃避攻击的攻击，在这种攻击中，攻击者对正常示例进行了少量，精心制作的更改，以欺骗模型做出错误的决策。 该项目的目标是通过表征对抗性和非对抗性实例之间的关系，对这些漏洞进行正式理解和防御能力，开发出利用这种关系以更好地检测对抗性示例的机制，指标和方法以证明机器学习模型对他们的鲁棒性。 共同开发的理论，算法和指标将共同提高机器学习系统的鲁棒性，从而使它们在关键任务应用程序中更牢固地部署。 该团队还将公开提供他们的数据集和源代码，并在自己的课程中使用它们，并与研究生和本科生进行研究，并特别努力包括来自代表性不足的科学，技术，工程，工程和数学的学生。 这项工作还将支持高中外展计划和夏令营，以吸引年轻的学生学习机器学习，安全性和计算机科学。该项目围绕着三个主要的推力组织，它们结合起来，为逃避攻击提供了整体方法，以提供建模和防御。 第一个推力旨在通过系统的测量研究来表征正常和对抗性示例。 这包括考虑特定示例周围的不同类型的区域（例如，度量球，歧管和转换引起的区域），然后根据许多算法来表征示例的漏洞，以结合附近地区其他示例的分类。 第二个推力重点是通过使用区域中的代表性数据点，汇总多个数据点并使用一组不同的分类器来设计对对抗性示例的强大防御，并使用单个数据点或算法来减少引起的漏洞。 第三个推力涉及定义用于建模鲁棒性以及利用这些指标来分析模型鲁棒性的理论和算法的指标。 这些包括公制球中对抗性扰动的下限，基于计算成本的稳健性度量，分析新数据集相对于培训数据的代表性的分析以及利用人类对对抗性估计的方法。该奖项反映了NSF的法定任务，并通过评估范围来评估支持者，并通过评估范围来进行基金会的范围。

项目成果

Protect privacy of deep classification networks by exploiting their generative power

• DOI: 10.1007/s10994-021-05951-6
• 发表时间: 2021-04
• 期刊: Machine Learning
• 影响因子: 7.5
• 作者: Jiyu Chen;Yiwen Guo;Qianjun Zheng;Hao Chen

Less is More: Culling the Training Set to Improve Robustness of Deep Neural Networks

少即是多：剔除训练集以提高深度神经网络的鲁棒性

• DOI: 10.1007/978-3-030-01554-1_6
• 发表时间: 2018
• 期刊: International Conference on Decision and Game Theory for Security
• 作者: Liu, Yongshuai;Chen, Jiyu;Chen, Hao
• 通讯作者: Chen, Hao

Fooling Detection Alone is Not Enough: First Adversarial Attack against Multiple Object Tracking

• 发表时间: 2019-05
• 期刊: ArXiv
• 作者: Yunhan Jia;Yantao Lu;Junjie Shen;Qi Alfred Chen;Zhenyu Zhong;Tao Wei

Integrity: Finding Integer Errors by Targeted Fuzzing

• DOI: 10.1007/978-3-030-63086-7_20
• 发表时间: 2020
• 作者: Yuyang Rong;Peng Chen;Hao Chen

Backpropagating Linearly Improves Transferability of Adversarial Examples

• 发表时间: 2020-12
• 期刊: ArXiv
• 作者: Yiwen Guo;Qizhang Li;Hao Chen