Speech privacy protection by high-quality, invertible, and extendable speech anonymization and de-anonymization

通过高质量、可逆、可扩展的语音匿名化和去匿名化保护语音隐私

• 批准号: 21K17775
• 负责人: Wang Xin
• 金额: $ 2.91万
• 依托单位: National Institute of Informatics
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Early-Career Scientists
• 财政年份: 2021
• 资助国家: 日本
• 起止时间: 2021-04-01 至 2024-03-31
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-21K17775/
• 关键词: speech privacy; speaker anonymization; speech waveform modeling; neural network; deep learning

The second year's work consists of three parts: Part 1) Based on the previous year's work, the second VoicePrivacy challenge was organized by us and other universities. We defined new evaluation frameworks and conducted solid evaluations. In addition to many findings, we found that the new baseline, which was the research outcome of the previous year, outperformed the legacy baseline. We also saw submissions that outperformed the new baseline, which indicates the advancement of the research field brought by the VoicePrivacy challenge.Part 2) Based on the framework of the voice privacy challenge, we did a deep analysis of the common approaches to generating anonymized speaker identity representation (i.e., pseudo speaker embedding). Through a large-scale experiment, we identified good strategies to choose and assign the pseudo-speaker, including random gender selection and utterance-level anonymization. We also found that a simple percentile-based pitch conversion reduced the risk against the strongest (Semi-Informed) attacker. These findings were published in a top IEEE journal.Part 3) We followed the research plan and extended the language-independent speaker anonymization framework. Although the framework is language-independent, its performance degrades when processing unseen languages. We found that using multilingual training data for the waveform generator was helpful. We also proposed a correlation-alignment-based strategy to alleviate channel mismatch. Additionally, we extended the framework to hide gender information. Both works were published in top conferences.

第二年的工作由三个部分组成： 第1部分）在前一年工作的基础上，我们和其他大学组织了第二届VoicePrivacy挑战赛。我们定义了新的评估框架并进行了扎实的评估。除了许多发现之外，我们还发现新基线（即前一年的研究成果）优于旧基线。我们还看到了优于新基线的提交，这表明了语音隐私挑战带来的研究领域的进步。第2部分）基于语音隐私挑战的框架，我们对生成匿名说话人的常见方法进行了深入分析身份表示（即伪说话者嵌入）。通过大规模实验，我们确定了选择和分配伪说话者的良好策略，包括随机性别选择和话语级匿名化。我们还发现，简单的基于百分位数的音高转换降低了针对最强（半知情）攻击者的风险。这些研究结果发表在IEEE顶级期刊上。第3部分）我们按照研究计划扩展了与语言无关的说话者匿名化框架。尽管该框架与语言无关，但在处理不可见的语言时其性能会下降。我们发现对波形发生器使用多语言训练数据很有帮助。我们还提出了一种基于相关对齐的策略来缓解通道不匹配。此外，我们扩展了框架以隐藏性别信息。两部作品均在顶级会议上发表。

项目成果

Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation

• DOI: 10.21437/odyssey.2022-16
• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Hemlata Tak;M. Todisco;Xin Wang;Jee-weon Jung;J. Yamagishi;N. Evans

Benchmarking and challenges in security and privacy for voice biometrics

语音生物识别安全和隐私方面的基准测试和挑战

• 发表时间: 2021
• 作者: Jean-Francois Bonastre; Hector Delgado; Nicholas Evans; Tomi Kinnunen; Kong Aik Lee; Xuechen Liu;Andreas Nautsch; Paul-Gauthier NoE;Jose Patino; Md Sahidullah; Brij Mohan Lal Srivastava;Massimiliano Todisco; Natalia Tomashenko; Emmanuel Vince
• 通讯作者: Emmanuel Vince

Languange-independent speaker anonymization system

与语言无关的说话者匿名系统

Tutorial on speaker anonymization (software)

说话者匿名化教程（软件）

Hiding Speaker’s Sex in Speech Using Zero-Evidence Speaker Representation in an Analysis/Synthesis Pipeline

• DOI: 10.1109/icassp49357.2023.10096749
• 发表时间: 2022-11
• 期刊: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Paul-Gauthier Noé;Xiaoxiao Miao;Xin Wang;J. Yamagishi;J. Bonastre;D. Matrouf