Collaborative Research: SHF: Medium: Toward Understandability and Interpretability for Neural Language Models of Source Code

合作研究：SHF：媒介：实现源代码神经语言模型的可理解性和可解释性

• 批准号: 2311468
• 负责人: Kevin Moran
• 金额: $ 74.52万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2024-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311468&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Toward

Advances in artificial intelligence (AI) have led to the development of several new types of tools for software developers that aim to help automate various parts of the software development process of building and maintaining software. However, the combination of complex underlying deep-learning models and massive training datasets makes it difficult to interpret why these models, and the developer tools powered by them, behave the way they do. Given the increasingly important role that these tools are beginning to play in software engineering (SE), it is imperative that techniques be developed that allow stakeholders to better understand and work with these tools such that critical software infrastructure can be maintained. This project will develop a framework and methodology that enables both researchers who build AI-powered developer tools, and software engineers who use these tools, to interpret why the underlying models make the predictions they do. The objective is to allow researchers to obtain detailed insights into why a model may not be performing as expected, allowing for targeted improvement and informed creation of new models. The methodology will be integrated into AI-powered software development tools, allowing software engineers to make informed decisions about when a tool’s suggestion may be helpful or harmful, thus building trust in their use. The interpretability framework will also enable new forms of interaction with these tools, providing a mechanism for natural language feedback that improves over time. This project will produce and disseminate educational materials on best practices related to building and using AI-powered programming tools. These materials are intended to be integrated into existing computer-literacy courses at all levels of education. In addition, the project will focus on recruiting and retaining computer science students from traditionally underrepresented categories.This project has three specific goals. First, it will design an automated approach for generating global explanations of the behavior of “context-free” neural language models for source code. This component of the project will map predictions from large language models to human-interpretable programming language concepts using causal inference theory, wherein explanations of behavior will be generated via causal interventions. Second, it will develop automated techniques for local explanations of contextualized language models of code by developing a set of interpretability techniques that generate behavioral, feature-based, and textual explanations defined for given SE tasks (e.g., program repair). Finally, the project will create techniques that enable researchers and developers to provide feedback to models based on generated explanations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）的进步促进了软件开发人员开发出多种新型工具，旨在帮助构建和维护软件的软件开发过程的各个部分实现自动化。然而，复杂的底层深度学习模型的组合。鉴于这些工具开始在软件工程 (SE) 中发挥越来越重要的作用，大量的训练数据集使得解释这些模型以及由它们提供支持的开发工具的行为方式变得困难。开发技术，使利益相关者能够更好地理解和使用这些工具该项目将开发一个框架和方法，使构建人工智能驱动的开发工具的研究人员和使用这些工具的软件工程师能够解释底层模型做出预测的原因。目标是让研究人员能够详细了解模型为何无法按预期运行，从而有针对性地改进并明智地创建新模型。该方法将集成到人工智能驱动的软件开发工具中，从而使软件工程师能够做出明智的决策。决定工具的建议何时可能有用或有害，从而建立信任可解释性框架还将支持与这些工具的新形式的交互，提供一种随着时间的推移而改进的自然语言反馈机制，并传播有关构建和使用人工智能编程工具的最佳实践的教育材料。这些材料旨在融入各级教育的现有计算机素养课程中。此外，该项目将侧重于招募和留住传统上代表性不足的类别的计算机科学学生。该项目有三个具体目标。设计一种自动化方法来生成全局对源代码的“上下文无关”神经语言模型的行为的解释 该项目的这个组件将使用因果推理理论将大型语言模型的预测映射到人类可解释的编程语言概念，因此将通过因果关系生成行为的解释。其次，它将通过开发一套行为可解释性技术来开发代码上下文语言模型的本地解释的自动化技术，这些技术为给定的 SE 任务（例如程序修复）生成基于特征的文本解释。将创造使研究人员和开发人员能够根据生成的解释向模型提供反馈的技术。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。