SaTC: CORE: Medium: Hidden Rules in Neural Networks as Attacks and Adversarial Defenses

SaTC：核心：中：神经网络中作为攻击和对抗性防御的隐藏规则

• 批准号: 1949650
• 负责人: Ben Zhao
• 金额: $ 120万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-03-01 至 2024-02-29
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1949650&HistoricalAwards=false
• 关键词: SaTC; CORE; Medium; Hidden; Rules

Recent advances in Deep Neural Networks (DNNs) have enabled significant progress in technological challenges such as voice/facial recognition, language translation and image recognition. Yet DNNs remain vulnerable to a class of hidden attacks called "backdoor" or "Trojan" attacks, where hidden rules are trained into a model which only become active on model input with some unusual properties, comprising a "trigger." They are strong enough that the presence of a small, inconspicuous trigger can make the model produce unexpected (and often erroneous) results, e.g., recognize anyone with a black ankh tattoo as a predetermined celebrity. Despite recent efforts, these attacks remain poorly understood, and robust defenses remain elusive. This project studies this class of attacks in depth to understand their potential impact on real machine learning systems and potential defenses.More specifically, the project will first catalog the breadth of backdoor attacks across multiple domains (and potential defenses), including images (facial and object recognition), text (natural language processing and sentiment analysis), and audio (speaker recognition and voice transcription). The project will then explore their practical implications outside the digital domain, including backdoor attacks in the physical world (such as on facial recognition), and advanced backdoors that coexist with transfer learning, the prevailing method for sharing DNN models today. Finally, the project will explore potential positive uses of backdoors as model-training tools, spawning a novel protection mechanism for DNN models, by trapping adversarial attacks with honey-pots built using backdoor techniques. The techniques will incorporate evaluation of both advanced attacks and defenses across a broad range of applications, datasets and models, and whenever possible, experiments in the physical domain. Successful results from this project should alert security professionals to the risk of backdoors in DNNs, while providing the software and algorithmic tools necessary for robust defenses.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络 (DNN) 的最新进展使得语音/面部识别、语言翻译和图像识别等技术挑战取得了重大进展。然而，DNN 仍然容易受到称为“后门”或“特洛伊木马”攻击的一类隐藏攻击的攻击，其中隐藏规则被训练到模型中，该模型仅在具有某些不寻常属性（包括“触发器”）的模型输入上才会激活。它们足够强大，以至于一个小的、不显眼的触发器的存在可以使模型产生意想不到的（并且经常是错误的）结果，例如，将任何有黑色安克纹身的人识别为预定的名人。尽管最近做出了努力，但人们对这些攻击仍然知之甚少，强大的防御措施仍然难以捉摸。 该项目深入研究此类攻击，以了解它们对真实机器学习系统和潜在防御的潜在影响。更具体地说，该项目将首先对跨多个领域（和潜在防御）的后门攻击的广度进行分类，包括图像（面部和图像）对象识别）、文本（自然语言处理和情感分析）和音频（说话人识别和语音转录）。然后，该项目将探索其在数字领域之外的实际影响，包括物理世界中的后门攻击（例如面部识别），以及与迁移学习（当今共享 DNN 模型的流行方法）共存的高级后门。最后，该项目将探索后门作为模型训练工具的潜在积极用途，通过使用后门技术构建的蜜罐捕获对抗性攻击，为 DNN 模型产生一种新颖的保护机制。这些技术将结合对广泛应用、数据集和模型的高级攻击和防御的评估，并尽可能在物理领域进行实验。该项目的成功结果应提醒安全专业人员注意 DNN 中的后门风险，同时提供稳健防御所需的软件和算法工具。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和技术进行评估，被认为值得支持。更广泛的影响审查标准。

项目成果

Finding Naturally Occurring Physical Backdoors in Image Datasets

在图像数据集中寻找自然发生的物理后门

• 发表时间: 2024-09-14
• 作者: Emily Wenger;Roma Bhattacharjee;A. Bhagoji;Josephine Passananti;Emilio Andere;Haitao Zheng;Ben Y. Zhao
• 通讯作者: Ben Y. Zhao

Fawkes: Protecting Privacy against Unauthorized Deep Learning Models

福克斯：保护隐私免受未经授权的深度学习模型的影响

• 发表时间: 2020-01
• 期刊: Proceedings of the 29th USENIX Security Symposium
• 作者: Shan, Shawn;Wenger, Emily;Zhang, Jiayun;Li, Huiying;Zheng, Haitao;Zhao, Ben Y.
• 通讯作者: Zhao, Ben Y.

"Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real World

“你好，是我”：现实世界中基于深度学习的语音合成攻击

• DOI: 10.1145/3460120.3484742
• 发表时间: 2021-11
• 期刊: CCS '21: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security
• 作者: Wenger, Emily;Bronckers, Ma;Cianfarani, Christian;Cryan, Jenna;Sha, Angela;Zheng, Haitao;Zhao, Ben Y.
• 通讯作者: Zhao, Ben Y.

Data Isotopes for Data Provenance in DNNs

DNN 中数据来源的数据同位素

• DOI: 10.48550/arxiv.2208.13893
• 发表时间: 2022-08-29
• 期刊: Proc. Priv. Enhancing Technol.
• 作者: Emily Wenger;Xiuyu Li;Ben Y. Zhao;Vitaly Shmatikov
• 通讯作者: Vitaly Shmatikov

User Authentication via Electrical Muscle Stimulation

通过肌肉电刺激进行用户身份验证

• DOI: 10.1145/3411764.3445441
• 发表时间: 2021-05-06
• 期刊: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
• 作者: Yuxin Chen;Zhuolin Yang;Ruben Abbou;Pedro Lopes;Ben Y. Zhao;Haitao Zheng
• 通讯作者: Haitao Zheng