EAGER: SaTC: SAVED: Secure Audio and Video Data from Deepfake Attacks Leveraging Environmental Fingerprints

EAGER：SaTC：SAVED：利用环境指纹保护音频和视频数据免遭 Deepfake 攻击

• 批准号: 2039342
• 负责人: Yu Chen
• 金额: $ 25.7万
• 依托单位: SUNY at Binghamton
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2039342&HistoricalAwards=false
• 关键词: EAGER; SaTC; SAVED; Secure; Audio

The fast development of artificial intelligence (AI) and machine learning algorithms is escalating the technology that empowers the ability to distort reality. It has taken an exponential leap forward to deepfake attacks, which create audio and video of real people saying and doing things they never said or did. It is ever more realistic and increasingly resistant to detection. Deepfaked video, audio, or photos published on social media platforms are highly disturbing and able to mislead the public, raising further challenges in policy, technology, social, and legal aspects. Today's deepfake tools allow people to become anyone, from Elon Musk to Eminem, during a video chat. Recent deepfake video attacks on some public scenarios have raised more concerns. Disinformation may actually cause a disturbance in our society and ruin the foundation of trust. Government agencies like the U.S. Defense Advanced Research Projects Agency (DARPA) are concerned about losing the war against deepfake attacks that use the popular machine learning technique to automatically incorporate artificial components into existing video streams. The detailed technical routines and countermeasures against deepfake attacks have not been well investigated, leaving alone a potentially effective approach to tackle the emerging threats online in real-time.This project introduces a novel solution to secure audio and video data streams against deepfake attacks. Instead of engaging in the endless AI arm races that fight fire with fire, where new machine learning algorithms keep making fake audio and video more real, this project tackles the challenging problem out of the box based on a key observation. Every audio or video stream has unique environmental fingerprints, e.g. the Electrical Network Frequency (ENF) signals, embedded when it was generated. The environmental fingerprints are random signals, which are unique, unpredictable, and unrepeatable. This project will investigate three typical application scenarios: (1) an accurate detection of deepfaked AVS data uploaded on the Internet, like social media posts; (2) an instant and accurate detection of false AVS injection attacks against online, real-time applications, like teleconferencing; and (3) a lightweight but robust version that fits on the Internet of Video Things applications, like smart public safety surveillance, which requires instant decision-making at the network edge. In addition, this project will gain deeper insights into the characteristics of the environmental fingerprints taking an information theory approach. The success of this research will deliver a disruptive technology that enables the ultimate win of the battle against the deepfake attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能 (AI) 和机器学习算法的快速发展正在使技术不断升级，从而赋予扭曲现实的能力。深度造假攻击已经取得了指数级的飞跃，深度造假攻击会创建真实的人所说和所做的事情的音频和视频，而他们从未说过或做过的事情。它变得更加真实，并且越来越难以被发现。在社交媒体平台上发布的深度伪造视频、音频或照片非常令人不安，并且能够误导公众，给政策、技术、社会和法律方面带来进一步的挑战。如今的深度换脸工具可以让人们在视频聊天中变成任何人，从埃隆·马斯克到埃米纳姆。最近针对一些公共场景的深度伪造视频攻击引发了更多担忧。虚假信息实际上可能会引起社会骚乱并破坏信任的基础。美国国防高级研究计划局 (DARPA) 等政府机构担心在针对 Deepfake 攻击的战争中失败，这些攻击使用流行的机器学习技术自动将人工组件合并到现有视频流中。针对 Deepfake 攻击的详细技术流程和对策尚未得到充分研究，更没有找到一种潜在有效的方法来实时应对在线新兴威胁。该项目引入了一种新颖的解决方案，以保护音频和视频数据流免受 Deepfake 攻击。该项目并没有参与无休止的以牙还牙的人工智能军备竞赛，新的机器学习算法不断使虚假的音频和视频变得更加真实，而是根据关键观察结果解决了开箱即用的挑战性问题。每个音频或视频流都有独特的环境指纹，例如生成时嵌入的电网频率 (ENF) 信号。环境指纹是随机信号，具有唯一性、不可预测性、不可重复性。该项目将研究三个典型的应用场景：（1）准确检测互联网上上传的深度伪造AVS数据，例如社交媒体帖子； (2) 即时准确地检测针对在线实时应用程序（例如电话会议）的虚假 AVS 注入攻击； (3) 轻量级但强大的版本，适合视频物联网应用，例如智能公共安全监控，需要在网络边缘进行即时决策。此外，该项目将采用信息论方法更深入地了解环境指纹的特征。这项研究的成功将提供一种颠覆性技术，最终赢得对抗深度造假攻击的战斗。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Fairledger: a Fair Proof-of-Sequential-Work based Lightweight Distributed Ledger for IoT Networks

Fairledger：基于公平顺序工作证明的物联网网络轻量级分布式账本

• DOI: 10.1109/blockchain55522.2022.00055
• 发表时间: 2022
• 期刊: the 5th IEEE International Conference on Blockchain (Blockchain 2022
• 作者: Xu, Ronghua;Chen, Yu
• 通讯作者: Chen, Yu

DeFakePro: Decentralized Deepfake Attacks Detection Using ENF Authentication

• DOI: 10.1109/mitp.2022.3172653
• 发表时间: 2022-07
• 期刊: IT Professional
• 影响因子: 2.6
• 作者: Deeraj Nagothu;Ronghua Xu;Yu Chen;E. Blasch;Alexander J. Aved

Detecting Compromised Edge Smart Cameras using Lightweight Environmental Fingerprint Consensus

使用轻量级环境指纹共识检测受损的边缘智能相机

• DOI: 10.1145/3485730.3493684
• 发表时间: 2021
• 期刊: The 19th ACM Conference on Embedded Networked Sensor Systems
• 作者: Nagothu, Deeraj;Xu, Ronghua;Chen, Yu;Blasch, Erik;Aved, Alexander
• 通讯作者: Aved, Alexander

Robustness of Electrical Network Frequency Signals as a Fingerprint for Digital Media Authentication

• DOI: 10.1109/mmsp55362.2022.9949315
• 发表时间: 2022-01-01
• 期刊: 2022 IEEE 24TH INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP)
• 作者: Poredi, Nihal;Nagothu, Deeraj;Blasch, Erik
• 通讯作者: Blasch, Erik

Deterring Deepfake Attacks with an Electrical Network Frequency Fingerprints Approach

• DOI: 10.3390/fi14050125
• 发表时间: 2022-04
• 期刊: Future Internet
• 影响因子: 3.4
• 作者: Deeraj Nagothu;Ronghua Xu;Yu Chen;E. Blasch;Alexander J. Aved