Collaborative Research: SaTC: CORE: Small: Detecting and Localizing Non-Functional Vulnerabilities in Machine Learning Libraries

协作研究：SaTC：核心：小型：检测和本地化机器学习库中的非功能性漏洞

• 批准号: 2230061
• 负责人: Gang Tan
• 金额: $ 24.66万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2230061&HistoricalAwards=false
• 关键词: Collaborative; Research; SaTC; CORE; Small

This project aims to improve security and resilience of machine learning (ML) software. Machine learning has been deployed in many critical domains such as drug discovery, financial planning, autonomous driving, and malware detection. This makes it crucial for ML-based software solutions to function properly even when attacked by malicious actors, leading to a line of research focused on functional vulnerabilities, attacks that attempt to make ML systems produce incorrect results. Less studied, however, are other kinds of vulnerabilities that don’t attack the core prediction functionality but still pose security risks. These “non-functional” vulnerabilities include denial of service attacks, which attempt to render the system unusable through overloading it; and side-channel attacks, which analyze features like response time to infer sensitive information about the models or data they are trained on. This project will develop methods for detecting and correcting these kinds of non-functional vulnerabilities and make those methods widely available, as well as disseminate educational materials to help security researchers and ML software developers be more aware of these risks. Despite a growing number of reported denial-of-service (DoS) and side channel (SC) vulnerabilities in core ML libraries such as NumPy and TensorFlow, a systematic approach to identifying and debugging them has not been explored due to multiple technical challenges: i) non-functional behaviors are not explicitly encoded in the syntax or semantics of ML code; ii) existing fault localization methods often fail to establish causal relationships; and iii) automatic DoS/SC mitigation is largely lacking for ML applications. This project will develop a novel methodology that combines evolutionary algorithms with a gradient-based guidance to detect DoS and quantify the strengths of SC vulnerabilities. For debugging, the project explores causally guided statistical methods to localize the root causes and guide an optimal mitigation policy. The project team will make a concerted effort to increase participation of women, Hispanic, and other underrepresented communities via special topic courses, research experiences for undergraduates, and summer camps for K-12 students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在提高机器学习 (ML) 软件的安全性和弹性。机器学习已部署在药物发现、财务规划、自动驾驶和恶意软件检测等许多关键领域，这对于基于 ML 的软件解决方案至关重要。即使受到恶意攻击者的攻击也能正常运行，从而导致一系列研究集中在功能漏洞上，而试图使机器学习系统产生错误结果的攻击则较少研究，而其他类型的漏洞则不会攻击核心预测。功能，但仍然存在安全风险。 “非功能性”漏洞包括拒绝服务攻击，这种攻击试图通过超载使系统无法使用；以及旁道攻击，这种攻击会分析响应时间等特征，以推断有关其所训练的模型或数据的敏感信息。该项目将开发检测和纠正此类非功能性漏洞的方法，并使这些方法得到广泛应用，并传播教育材料，以帮助安全研究人员和机器学习软件开发人员更加了解这些风险，尽管报告的数量越来越多。 NumPy 和 TensorFlow 等核心 ML 库中存在拒绝服务 (DoS) 和侧通道 (SC) 漏洞，但由于存在多项技术挑战，尚未探索出识别和调试这些漏洞的系统方法：i) 非功能性行为没有明确编码在 ML 代码的语法或语义中；ii) 现有的故障定位方法通常无法建立因果关系；以及 iii) ML 应用程序很大程度上缺乏自动 DoS/SC 缓解。该项目将进化算法与基于梯度的指导相结合来检测 DoS 并量化 SC 漏洞的强度。为了进行调试，该项目探索因果引导的统计方法来定位根本原因并指导最佳的缓解策略。通过专题课程、本科生研究经验和 K-12 学生夏令营，共同努力增加女性、西班牙裔和其他代表性不足社区的参与。该奖项是 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准。