RI: Small: Low-Latency and High-Quality Simultaneous Translation

RI：小：低延迟、高质量同声翻译

• 批准号: 2009071
• 负责人: Liang Huang
• 金额: $ 45万
• 依托单位: Oregon State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2009071&HistoricalAwards=false
• 关键词: RI; Small; Low; Latency; Quality

Simultaneous language translation (interpretation) is widely used in many situations including multilateral organizations such as the United Nations, international summits and conferences, and legal proceedings. However, the concurrent perception and production in two languages makes this task extremely challenging and exhausting for humans. The number of professional simultaneous interpreters is extremely limited worldwide, and they have to work in groups of two or three where each interpreter can only sustain for about 15-30 minutes. Therefore, there is a critical need to develop simultaneous translation techniques to reduce the burden of human interpreters and make this service more accessible and affordable. However, simultaneous translation is also notoriously difficult for machines and accomplishing it consistently and reliably is considered one of the holy grails of Artificial Intelligence. Various methods have been proposed to solve this problem, but with three major limitations: (a) their translation model is still a full-sentence translation model; (b) they cannot achieve short latencies such as "3-seconds delay" common in human interpretation; and (c) their systems are complicated and difficult to train. Therefore, this project aims to develop new algorithms, techniques, and datasets for high-quality simultaneous machine translation with minimum delay (low latency). The technologies developed by this project will make simultaneous translation more affordable and accessible, which will improve the efficiency of human communication across linguistic barriers. This project also supports STEM education of underrepresented minorities (who do not speak English natively) by recruiting them in machine translation studies.Based on the principal investigator's successful prior work, the key idea in this project is to discard the conventional full-sentence translation paradigm and the classical sequence-to-sequence framework which processes the full input sentence before starting to translate and are thus ill-suited to simultaneous translation. Instead, this project adopts a "prefix-to-prefix" framework which starts translation after processing only a few input words, mimicking human interpreters. Though extremely simple, this framework achieves low latency and high translation quality. Using this framework, this project aims to (1) Develop an algorithm to detect and fix anticipation mistakes on the fly, and explore new evaluation metrics that can work for translations with revisions; (2) Develop dynamic and flexible translation strategies to balance quality and latency; (3) Construct better training data for simultaneous translation by revising the reference translations in a parallel text to remove unnecessary reorderings; (4) Apply the prefix-to-prefix framework to incremental text-to-speech synthesis (TTS), thus completing the end-to-end simultaneous speech-to-speech pipeline, improve its quality and latency, and compare with human simultaneous interpreters.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

同声传译（口译）广泛应用于联合国等多边组织、国际峰会和会议、法律诉讼等多种场合。然而，两种语言的同时感知和产生使得这项任务对人类来说极具挑战性和疲惫不堪。全球范围内专业同声传译员的数量极其有限，而且他们必须以两到三人一组的方式工作，每个译员只能持续大约15-30分钟。因此，迫切需要开发同声翻译技术，以减轻人工口译员的负担，并使这项服务更容易获得和负担得起。然而，同声翻译对于机器来说也是出了名的困难，一致且可靠地完成同声翻译被认为是人工智能的圣杯之一。人们提出了各种方法来解决这个问题，但存在三个主要局限性：（a）它们的翻译模型仍然是全句翻译模型； (b) 它们无法实现短延迟，例如人类解释中常见的“3秒延迟”； (c) 他们的系统复杂且难以训练。 因此，该项目旨在开发新的算法、技术和数据集，以实现最小延迟（低延迟）的高质量同步机器翻译。该项目开发的技术将使同声翻译变得更加经济实惠和易于使用，从而提高跨越语言障碍的人类交流效率。该项目还通过招募机器翻译研究来支持代表性不足的少数群体（母语不是英语）的 STEM 教育。基于主要研究者之前的成功工作，该项目的关键思想是放弃传统的整句翻译范式经典的序列到序列框架在开始翻译之前处理完整的输入句子，因此不适合同声翻译。相反，该项目采用“前缀到前缀”框架，在仅处理几个输入单词后开始翻译，模仿人类翻译。虽然非常简单，但该框架实现了低延迟和高翻译质量。使用该框架，该项目的目标是（1）开发一种算法来动态检测和修复预期错误，并探索可用于修订翻译的新评估指标； (2) 制定动态、灵活的翻译策略，平衡质量和延迟； （3）通过修改平行文本中的参考译文，删除不必要的重新排序，构建更好的同声翻译训练数据； (4) 将前缀到前缀框架应用于增量文本到语音合成（TTS），从而完成端到端同步语音到语音管道，提高其质量和延迟，并与人类同步进行比较该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

LazySampling and LinearSampling: fast stochastic sampling of RNA secondary structure with applications to SARS-CoV-2

LazySampling 和 LinearSampling：RNA 二级结构的快速随机采样及其在 SARS-CoV-2 中的应用

• DOI: 10.1093/nar/gkac1029
• 发表时间: 2022-11
• 期刊: Nucleic Acids Research
• 影响因子: 14.9
• 作者: Zhang, He;Li, Sizhen;Zhang, Liang;Mathews, David H.;Huang, Liang
• 通讯作者: Huang, Liang

Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR

同步流 ASR 辅助的直接同步语音到文本翻译

• DOI: 10.18653/v1/2021.findings-acl.406
• 发表时间: 2021-01
• 期刊: Proceedings of ACL 2021: Findings
• 作者: Chen, Junkun;Ma, Mingbo;Zheng, Renjie;Huang, Liang
• 通讯作者: Huang, Liang

Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings

通过合并伪引用并减少重新排序来改进同声翻译

• DOI: 10.18653/v1/2021.emnlp-main.473
• 发表时间: 2021-01
• 期刊: Proceedings of EMNLP 2021
• 作者: Chen, Junkun;Zheng, Renjie;Kita, Atsuhito;Ma, Mingbo;Huang, Liang
• 通讯作者: Huang, Liang

RNA design via structure-aware multifrontier ensemble optimization

通过结构感知多前沿集成优化进行 RNA 设计

• DOI: 10.1093/bioinformatics/btad252
• 发表时间: 2023-06-30
• 期刊: Bioinformatics
• 影响因子: 5.8
• 作者: Zhou, Tianshuo;Dai, Ning;Li, Sizhen;Ward, Ma;Mathews, David H.;Huang, Liang
• 通讯作者: Huang, Liang

LinearTurboFold: Linear-time global prediction of conserved structures for RNA homologs with applications to SARS-CoV-2

LinearTurboFold：RNA 同系物保守结构的线性时间全局预测及其在 SARS-CoV-2 中的应用

• DOI: 10.1073/pnas.2116269118
• 发表时间: 2021-12
• 期刊: Proceedings of the National Academy of Sciences
• 作者: Li, Sizhen;Zhang, He;Zhang, Liang;Liu, Kaibo;Liu, Boxiang;Mathews, David H.;Huang, Liang
• 通讯作者: Huang, Liang