EAGER: SaTC: SAVED: Secure Audio and Video Data from Deepfake Attacks Leveraging Environmental Fingerprints

EAGER：SaTC：SAVED：利用环境指纹保护音频和视频数据免遭 Deepfake 攻击

• 批准号: 2039342
• 负责人: Yu Chen
• 金额: $ 25.7万
• 依托单位: SUNY at Binghamton
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2039342&HistoricalAwards=false
• 关键词: EAGER; SaTC; SAVED; Secure; Audio

The fast development of artificial intelligence (AI) and machine learning algorithms is escalating the technology that empowers the ability to distort reality. It has taken an exponential leap forward to deepfake attacks, which create audio and video of real people saying and doing things they never said or did. It is ever more realistic and increasingly resistant to detection. Deepfaked video, audio, or photos published on social media platforms are highly disturbing and able to mislead the public, raising further challenges in policy, technology, social, and legal aspects. Today's deepfake tools allow people to become anyone, from Elon Musk to Eminem, during a video chat. Recent deepfake video attacks on some public scenarios have raised more concerns. Disinformation may actually cause a disturbance in our society and ruin the foundation of trust. Government agencies like the U.S. Defense Advanced Research Projects Agency (DARPA) are concerned about losing the war against deepfake attacks that use the popular machine learning technique to automatically incorporate artificial components into existing video streams. The detailed technical routines and countermeasures against deepfake attacks have not been well investigated, leaving alone a potentially effective approach to tackle the emerging threats online in real-time.This project introduces a novel solution to secure audio and video data streams against deepfake attacks. Instead of engaging in the endless AI arm races that fight fire with fire, where new machine learning algorithms keep making fake audio and video more real, this project tackles the challenging problem out of the box based on a key observation. Every audio or video stream has unique environmental fingerprints, e.g. the Electrical Network Frequency (ENF) signals, embedded when it was generated. The environmental fingerprints are random signals, which are unique, unpredictable, and unrepeatable. This project will investigate three typical application scenarios: (1) an accurate detection of deepfaked AVS data uploaded on the Internet, like social media posts; (2) an instant and accurate detection of false AVS injection attacks against online, real-time applications, like teleconferencing; and (3) a lightweight but robust version that fits on the Internet of Video Things applications, like smart public safety surveillance, which requires instant decision-making at the network edge. In addition, this project will gain deeper insights into the characteristics of the environmental fingerprints taking an information theory approach. The success of this research will deliver a disruptive technology that enables the ultimate win of the battle against the deepfake attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）和机器学习算法的快速发展正在逐步升级，使现实扭曲现实的能力升级。这已经取得了指数级的飞跃，攻击了Deepfake的攻击，这些攻击创造了真实的人说和做他们从未说过或做过的事情的视频。它越来越现实，越来越抵抗检测。在社交媒体平台上发布的深层视频，音频或照片非常令人不安，能够误导公众，从而在政策，技术，社会和法律方面提出了进一步的挑战。当今的DeepFake工具使人们可以在视频聊天期间成为任何人，从Elon Musk到Eminem。最近对某些公共场景的DeepFake视频攻击引起了更多的担忧。虚假信息实际上可能会导致我们的社会骚扰，并破坏信任的基础。像美国国防高级研究项目局（DARPA）这样的政府机构担心输掉反对深击攻击的战争，这些攻击使用流行的机器学习技术将人工组件自动纳入现有视频流。尚未对针对深冰攻击的详细技术例程和对策进行，仅对实时应对在线应对新兴威胁的潜在有效方法。该项目介绍了一种新的解决方案，以保护音频和视频数据流，以防止深击攻击。与其参加无休止的AI手臂与火灾作斗争的无尽的AI手臂，在这种情况下，新的机器学习算法不断使虚假的音频和视频更真实，而是根据关键观察来解决挑战性的问题。每个音频或视频流都有独特的环境指纹，例如电网络频率（ENF）信号在生成时嵌入。环境指纹是随机信号，它是独特的，不可预测的且无法重复的。该项目将调查三个典型的应用程序方案：（1）对上传到Internet上载的深层AVS数据的准确检测，例如社交媒体帖子； （2）对虚假的AVS注射攻击对在线，实时应用程序（如电信）的攻击； （3）适合在视频事物应用程序上使用的轻巧但可靠的版本，例如智能公共安全监视，它需要在网络边缘即时决策。此外，该项目将对采用信息理论方法的环境指纹的特征有了更深入的了解。这项研究的成功将提供一项破坏性的技术，使反对DeepFake攻击的战斗最终胜利。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准通过评估来获得支持的。

项目成果

Fairledger: a Fair Proof-of-Sequential-Work based Lightweight Distributed Ledger for IoT Networks

Fairledger：基于公平顺序工作证明的物联网网络轻量级分布式账本

• DOI: 10.1109/blockchain55522.2022.00055
• 发表时间: 2022
• 期刊: the 5th IEEE International Conference on Blockchain (Blockchain 2022
• 作者: Xu, Ronghua;Chen, Yu
• 通讯作者: Chen, Yu

DeFakePro: Decentralized Deepfake Attacks Detection Using ENF Authentication

• DOI: 10.1109/mitp.2022.3172653
• 发表时间: 2022-07
• 期刊: IT Professional
• 影响因子: 2.6
• 作者: Deeraj Nagothu;Ronghua Xu;Yu Chen;E. Blasch;Alexander J. Aved

Detecting Compromised Edge Smart Cameras using Lightweight Environmental Fingerprint Consensus

使用轻量级环境指纹共识检测受损的边缘智能相机

• DOI: 10.1145/3485730.3493684
• 发表时间: 2021
• 期刊: The 19th ACM Conference on Embedded Networked Sensor Systems
• 作者: Nagothu, Deeraj;Xu, Ronghua;Chen, Yu;Blasch, Erik;Aved, Alexander
• 通讯作者: Aved, Alexander

Robustness of Electrical Network Frequency Signals as a Fingerprint for Digital Media Authentication

• DOI: 10.1109/mmsp55362.2022.9949315
• 发表时间: 2022-01-01
• 期刊: 2022 IEEE 24TH INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP)
• 作者: Poredi, Nihal;Nagothu, Deeraj;Blasch, Erik
• 通讯作者: Blasch, Erik

Deterring Deepfake Attacks with an Electrical Network Frequency Fingerprints Approach

• DOI: 10.3390/fi14050125
• 发表时间: 2022-04
• 期刊: Future Internet
• 影响因子: 3.4
• 作者: Deeraj Nagothu;Ronghua Xu;Yu Chen;E. Blasch;Alexander J. Aved