III: Small: Collaborative Research: Scrutable and Explainable Information Retrieval with Model Intrinsic and Agnostic Approaches

III：小：协作研究：使用模型内在和不可知的方法进行可查和可解释的信息检索

• 批准号: 2007398
• 负责人: Vivek Srikumar
• 金额: $ 30万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007398&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; Scrutable

Information Retrieval (IR) systems are important for people for information access. For example, intelligent search engines are widely used in Web-based services such as web search, product search, and job search. Recently, sophisticated data and complicated black-box models have made modern IR systems less transparent to users. However, as more and more people rely on IR systems to guide their daily life and decision making, there has been growing needs of explainable search results, both for technical communities and the general public, so that they understand why certain search results are provided. Meanwhile, governmental agencies are demanding IR systems to provide not only high-quality results, but also reasonable justifications, so as to enhance the trustworthiness of the systems. This project focuses on developing algorithms and frameworks to improve the scrutability, explainability, and transparency of modern IR systems. It will inspire large-scale academic-industry collaboration, which benefits billions of users by facilitating the development of reliable and explainable information access services. This project will develop general and reusable frameworks for scrutable and explainable IR. Research in this project will be performed on two directions. The first direction aims at new retrieval models for model-intrinsic explanation. This includes developing transparent inference process and decision boundaries for retrieval actions, scrutable functions that support result exploration with user feedback, and traceable information flow to distinguish the contribution of model inputs. The second direction aims at building analytical and simulative framework for model-agnostic explanation. This includes post-hoc explanation systems with external knowledge, and a simulation framework over black-box retrieval models with explainable outputs. Besides model-intrinsic and model-agnostic approaches, this project will also investigate crowd-sourcing tasks and systematic metrics to compare the effectiveness of intrinsic and agnostic explanations. The research outcomes will include multiple public benchmark datasets and evaluation platforms for explainable IR, which will contribute to the research community for sustainable and reproducible future studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

信息检索（IR）系统对于信息访问的人很重要。例如，智能搜索引擎被广泛用于基于Web的服务，例如Web搜索，产品搜索和求职。最近，复杂的数据和复杂的黑盒模型使现代红外系统对用户的透明度较低。但是，随着越来越多的人依靠IR系统来指导其日常生活和决策，对于技术社区和公众来说，越来越多的可解释搜索结果的需求，以便他们了解为什么提供了某些搜索结果。同时，政府机构要求IR系统不仅提供高质量的结果，而且还提供合理的理由，以增强系统的可信度。该项目着重于开发算法和框架，以提高现代IR系统的可仔可辨式性，解释性和透明度。它将激发大规模的学术行业合作，通过促进可靠且可解释的信息访问服务的开发来使数十亿用户受益。该项目将开发一般和可重复使用的框架，以供可审查和可解释的IR。该项目的研究将在两个方向上进行。第一个方向旨在实现用于模型内部解释的新检索模型。这包括为检索操作开发透明的推理过程和决策边界，可审查的功能支持用户反馈探索结果，以及可追溯的信息流以区分模型输入的贡献。第二个方向旨在构建模型不合时宜的解释的分析和模拟框架。这包括具有外部知识的事后解释系统，以及具有可解释输出的黑盒检索模型上的模拟框架。除了模型内在和模型不足的方法外，该项目还将研究众包任务和系统的指标，以比较内在和不可知论解释的有效性。研究成果将包括多个公共基准数据集和可解释的IR的评估平台，这将为可持续和可重现的未来研究研究社区做出贡献。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点通过评估来支持的。和更广泛的影响审查标准。

项目成果

Understanding the Effectiveness of Reviews in E-commerce Top-N Recommendation

• DOI: 10.1145/3471158.3472258
• 发表时间: 2021-06
• 期刊: Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval
• 作者: Zhichao Xu;Hansi Zeng;Qingyao Ai

Model-agnostic vs. Model-intrinsic Interpretability for Explainable Product Search

• DOI: 10.1145/3459637.3482276
• 发表时间: 2021-01-01
• 期刊: PROCEEDINGS OF THE 30TH ACM INTERNATIONAL CONFERENCE ON INFORMATION & KNOWLEDGE MANAGEMENT, CIKM 2021
• 作者: Ai, Qingyao;Narayanan, Lakshmi R.
• 通讯作者: Narayanan, Lakshmi R.