Unifying Pre-training and Multilingual Semantic Representation Learning for Low-resource Neural Machine Translation

统一预训练和多语言语义表示学习以实现低资源神经机器翻译

基本信息

批准号：
22KJ1843
负责人：
毛卓遠
金额：
$ 1.09万
依托单位：
Kyoto University
依托单位国家：
日本
项目类别：
Grant-in-Aid for JSPS Fellows
财政年份：
2023
资助国家：
日本
起止时间：
2023-03-08 至 2024-03-31
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-22KJ1843/
关键词：
multilingual translation low-resource translation multilingual embedding model efficiency

项目摘要

In the past year, we focused on improving the efficiency of multilingual sentence representation learning and exploring novel methods for improving multilingual machine translation. Both research promotes the research for multilingual / low-resource neural machine translation.(1) We proposed an efficient and effective method for training and presented the work in 言語処理学会 2023. On the other hand, we proposed knowledge distillation for compressing a large model, and it has been accepted to EACL 2023 main conference, which leads to efficient model inference. With the above achievements, the process of collecting parallel sentences for training translation systems will be accelerated. Specifically, the model training phase can be accelerated by 4 - 16 times, and the model inference phase can achieve 2.5 - 5 times speedup with further faster speed on downstream tasks.(2) We explored novel ways to improve the multilingual translation system with a word-level contrastive learning technique and obtained better translation quality for low-resource language pairs, which was accepted by NAACL 2022 findings. We also explained the improvements by showing the relationship between BLEU scores and sentence retrieval performance of the NMT encoder, which motivates that future work can focus on further improving the encoder’s retrieval performance in many-to-many NMT and contrastive objective’s feasibility in a massively multilingual scenario.

在过去的一年里，我们专注于提高多语言句子表示学习的效率并探索改进多语言机器翻译的新方法，这两项研究都促进了多语言/低资源神经机器翻译的研究。（1）我们提出了一种高效且有效的方法。方法进行训练，并在日本语言处理学会 2023 上展示了工作。另一方面，我们提出了压缩大型模型的知识蒸馏，并已被 EACL 2023 主会议接受，从而产生了高效的模型有了上述成果，收集并行句子用于训练翻译系统的过程将加速，模型训练阶段可以加速4-16倍，模型推理阶段可以实现2.5-5倍的加速。 (2)我们探索了利用词级对比学习技术改进多语言翻译系统的新方法，并为低资源语言对获得了更好的翻译质量，该结果被NAACL 2022研究结果所接受。还通过展示 NMT 编码器的 BLEU 分数和句子检索性能之间的关系来解释这些改进，这促使未来的工作可以集中于进一步提高编码器在多对多 NMT 中的检索性能以及对比目标在大规模多语言场景中的可行性。