III: Small: Efficiency Optimization for Neural Document Ranking with Compact Representations

III：小：具有紧凑表示的神经文档排序的效率优化

• 批准号: 2225942
• 负责人: Tao Yang
• 金额: $ 59.37万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2225942&HistoricalAwards=false
• 关键词: III; Small; Efficiency; Optimization; Neural

Over the last few years, the resurgence of neural models has greatly advanced the field of information retrieval enabling retrieval engines to effectively match and rank search results in response to a user query. For example, this new technology has enable to determine the most relevant documents in response to a query even when some query keywords may not appear in these documents. The main drawback of using deep neural models for ranking is that the retrieval is extremely time consuming. As a result, such models cannot be deployed in many practical search applications. This project is focused on studying efficient solutions to perform neural ranking computation and the developed techniques will be evaluated using public datasets to assess the solution’s effectiveness. The project integrates the research with an educational plan including undergraduate and graduate students' involvement, instructional material development, and outreach activities. This project carries out a two-thrust research agenda for efficient neural ranking. The first thrust investigates a fast re-ranking scheme for a dual-encoding architecture by leveraging precomputed embeddings to compose a query representation with approximation, and combining deep contextual token interactions and traditional lexical matching features. The second thrust of this project investigates a compact representation of document embeddings and strike a balance of relevance and space efficiency which affects online inference latency. The project exploits the composite nature of ranking inference for answering a query to approximate query embeddings, and decouples ranking contribution of document embeddings in deriving a compact representation. This research will advance our fundamental understanding of relevance and efficiency tradeoffs in neural information retrieval, and significantly reduce the computing and space cost of online inference while retaining the essential benefits of deep learning for effective ranking on affordable computing platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去几年中，神经模型的复兴极大地推进了信息检索领域，使检索引擎能够根据用户查询有效地匹配和排名搜索结果，例如，这项新技术能够确定最相关的文档。即使某些查询关键字可能不会出现在这些文档中，也要响应查询。使用深度神经模型进行排名的主要缺点是检索非常耗时，因此，此类模型无法部署在许多实际搜索应用中。 .这个项目专注于研究执行神经排名计算的有效解决方案，并将使用公共数据集对所开发的技术进行评估，以评估集成解决方案的有效性。该项目的研究包括本科生和研究生的参与、教学材料的开发和推广。该项目开展了高效神经排序的两个重点研究议程，第一个重点是通过利用预先计算的嵌入来构建具有近似值的查询表示，研究双编码架构的快速重新排序方案。该项目的第二个重点是研究文档嵌入的紧凑表示，并在影响在线推理延迟的相关性和空间效率之间取得平衡。回答查询以近似查询嵌入，并在导出紧凑表示时解耦文档嵌入的排名贡献。这项研究将增进我们对神经信息检索中的相关性和效率权衡的基本理解。显着降低在线推理的计算和空间成本，同时保留深度学习的基本优势，以便在负担得起的计算平台上进行有效排名。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查进行评估，被认为值得支持标准。

项目成果

Balanced Knowledge Distillation with Contrastive Learning for Document Re-ranking

文档重排序的对比学习平衡知识蒸馏

• DOI: 10.1145/3578337.3605120
• 发表时间: 2023-08-09
• 期刊: Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval
• 作者: Yingrui Yang;Shanxiu He;Yifan Qiao;Wentai Xie;Tao Yang
• 通讯作者: Tao Yang

Optimizing Guided Traversal for Fast Learned Sparse Retrieval

优化引导遍历以实现快速学习的稀疏检索

• DOI: 10.1145/3543507.3583497
• 发表时间: 2023-04
• 期刊: Proc. of the ACM Web Conference 2023 (WWW ’23
• 作者: Qiao, Yifan;Yang, Yingrui;Lin, Haixin;Yang, Tao
• 通讯作者: Yang, Tao

Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval

具有混合阈值的表示稀疏化，用于基于 SPLADE 的快速文档检索

• DOI: 10.1145/3539618.3592051
• 发表时间: 2023-06-20
• 期刊: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
• 作者: Yifan Qiao;Yingrui Yang;Shanxiu He;Tao Yang
• 通讯作者: Tao Yang