Collaborative Research: RI: Medium: Multilingual Long-form QA with Retrieval-Augmented Language Models

合作研究：RI：Medium：采用检索增强语言模型的多语言长格式 QA

• 批准号: 2312949
• 负责人: Mohit Iyyer
• 金额: $ 55.42万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2027-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2312949&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Medium; Multilingual

This project aims to enable automatic question answering systems to produce paragraph-level answers. Prior work on question answering has focused on simpler questions that can be answered with short phrases. Building systems to produce paragraph-level answers opens up exciting opportunities to answer complicated questions, and to offer more nuanced and comprehensive answers to simpler questions. This project will create comprehensive and reliable evaluation protocols for long form question answering (LFQA), pioneer multilingual studies to broaden information access to a wider population, and develop new algorithms that integrate web search with LFQA systems to provide verifiable long form answers paired with human-written evidence documents. This project focuses on three core dimensions of LFQA – datasets, evaluation, and modeling. Expanding the scope of prior English-centric LFQA, this research will investigate multilingual capabilities of large language models by constructing multilingual LFQA datasets and studying knowledge transfer across languages. In terms of modeling, it will propose a new framework that iteratively weaves together – in a transparent manner—knowledge retrieved from documents and memorized knowledge from a language model. Finally for evaluation, the project will engage domain experts who are familiar with the question topic to provide rationales for their evaluation of model generated answers. Such feedback will be used to derive a fine-grained annotation framework which localizes errors and unpack the weaknesses of generated answers. Together, the proposed work will bring significant progress to LFQA, an emerging topic for natural language processing and artificial intelligence research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在使自动问答系统能够产生段落级别的答案。关于问答的先前工作集中在可以用简短短语回答的更简单问题上。建立制作段落级别答案的系统为回答全面问题提供了令人兴奋的机会，并为更简单的问题提供了更多细微和全面的答案。该项目将为长期问答答案（LFQA）创建全面可靠的评估协议，先驱多语言研究，以扩大信息访问更广泛的人群的信息，并开发新的算法，这些算法将Web搜索与LFQA系统集成在一起，以提供可验证的长形式答案，并与人提供的证据配对。该项目着重于LFQA - 数据集，评估和建模的三个核心维度。扩大以前以英语为中心的LFQA的范围，这项研究将通过构建多语言LFQA数据集并研究跨语言的知识转移来研究大语言模型的多语言能力。在建模方面，它将提出一个新的框架，以透明的方式迭代地编织在一起，从文档中检索到的知识和从语言模型中的记忆知识中检索出来。最后，为了进行评估，该项目将吸引熟悉问题主题的领域专家，以提供对模型生成的答案的评估的理由。此类反馈将用于得出一个细粒的注释框架，该框架本地化错误并解开了生成的答案的弱点。拟议的工作将共同为LFQA带来重大进展，LFQA是一个新兴的自然语言处理和人工智能研究的主题。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响评估审查标准来评估，被认为是珍贵的支持。