Speech privacy protection by high-quality, invertible, and extendable speech anonymization and de-anonymization

通过高质量、可逆、可扩展的语音匿名化和去匿名化保护语音隐私

• 批准号: 21K17775
• 负责人: Wang Xin
• 金额: $ 2.91万
• 依托单位: National Institute of Informatics
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Early-Career Scientists
• 财政年份: 2021
• 资助国家: 日本
• 起止时间: 2021-04-01 至 2024-03-31
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-21K17775/
• 关键词: speech privacy; speaker anonymization; speech waveform modeling; neural network; deep learning

The second year's work consists of three parts: Part 1) Based on the previous year's work, the second VoicePrivacy challenge was organized by us and other universities. We defined new evaluation frameworks and conducted solid evaluations. In addition to many findings, we found that the new baseline, which was the research outcome of the previous year, outperformed the legacy baseline. We also saw submissions that outperformed the new baseline, which indicates the advancement of the research field brought by the VoicePrivacy challenge.Part 2) Based on the framework of the voice privacy challenge, we did a deep analysis of the common approaches to generating anonymized speaker identity representation (i.e., pseudo speaker embedding). Through a large-scale experiment, we identified good strategies to choose and assign the pseudo-speaker, including random gender selection and utterance-level anonymization. We also found that a simple percentile-based pitch conversion reduced the risk against the strongest (Semi-Informed) attacker. These findings were published in a top IEEE journal.Part 3) We followed the research plan and extended the language-independent speaker anonymization framework. Although the framework is language-independent, its performance degrades when processing unseen languages. We found that using multilingual training data for the waveform generator was helpful. We also proposed a correlation-alignment-based strategy to alleviate channel mismatch. Additionally, we extended the framework to hide gender information. Both works were published in top conferences.

第二年的工作由三个部分组成：第1部分）根据上一年的工作，第二个语音私人挑战是由美国和其他大学组织的。我们定义了新的评估框架并进行了稳固的评估。除了许多发现外，我们还发现，这是上一年的研究结果的新基线表现优于旧基线。我们还看到了表现优于新基线的提交，这表明语音挑战挑战带来的研究领域的进步。第2部分）基于语音隐私挑战的框架，我们对产生匿名说话者的常见方法进行了深入的分析（即，pseudo speaudo speader smerverding）。通过大规模的实验，我们确定了良好的策略来选择和分配伪演讲者，包括随机的性别选择和话语级别的匿名化。我们还发现，基于百分位的音高转换降低了对最强（半信息）攻击者的风险。这些发现发表在最高的IEEE期刊上。第3部分）我们遵循了研究计划，并扩展了与语言无关的说话者匿名框架。尽管该框架与语言无关，但在处理看不见的语言时，其性能会降低。我们发现，为波形生成器使用多语言培训数据很有帮助。我们还提出了一种基于相关的策略，以减轻渠道不匹配。此外，我们扩展了框架以隐藏性别信息。这两部作品均在顶级会议上发表。

项目成果

Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation

• DOI: 10.21437/odyssey.2022-16
• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Hemlata Tak;M. Todisco;Xin Wang;Jee-weon Jung;J. Yamagishi;N. Evans

Benchmarking and challenges in security and privacy for voice biometrics

语音生物识别安全和隐私方面的基准测试和挑战

• 发表时间: 2021
• 作者: Jean-Francois Bonastre; Hector Delgado; Nicholas Evans; Tomi Kinnunen; Kong Aik Lee; Xuechen Liu;Andreas Nautsch; Paul-Gauthier NoE;Jose Patino; Md Sahidullah; Brij Mohan Lal Srivastava;Massimiliano Todisco; Natalia Tomashenko; Emmanuel Vince
• 通讯作者: Emmanuel Vince

Tutorial on speaker anonymization (software)

说话者匿名化教程（软件）

Languange-independent speaker anonymization system

与语言无关的说话者匿名系统

Hiding Speaker’s Sex in Speech Using Zero-Evidence Speaker Representation in an Analysis/Synthesis Pipeline

• DOI: 10.1109/icassp49357.2023.10096749
• 发表时间: 2022-11
• 期刊: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Paul-Gauthier Noé;Xiaoxiao Miao;Xin Wang;J. Yamagishi;J. Bonastre;D. Matrouf