Collaborative Research: SHF: Medium: Natural Language Models with Execution Data for Software Testing

协作研究：SHF：媒介：用于软件测试的具有执行数据的自然语言模型

基本信息

批准号：
2313027
负责人：
Milos Gligoric
金额：
$ 90万
依托单位：
University of Texas at Austin
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-10-01 至 2027-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313027&HistoricalAwards=false
关键词：
Collaborative Research SHF Medium Natural

项目摘要

Natural Language Processing (NLP) models have proven useful for various software engineering tasks, including code completion, comment generation and update, code review generation, and clone detection. Despite the importance of software testing in industry, there has been little work on using these Artificial Intelligence (AI) models for developing and maintaining test code, which is a key part of software testing in the real world. Test code differs in multiple ways from regular code: (1) Test code is structured in a specific way, with steps for setting up a test environment and comparing expected results; (2) Test code has richer context, such as the specific methods and code it is testing (code under test); (3) Test code uses different code elements than the code under test, i.e., it has a different control structure; (4) Test code has specific input values and expected results; (5) Unlike regular code, test code can be readily executed.The goal of this project is to increase the productivity of software engineers via NLP models that simplify the development and maintenance of tests (NLP4Test). Specifically, tasks include test generation and completion, test update (when the underlying code changes), and automatically migrating tests across different programming languages. This project explores testing both general codebases and emerging machine learning (ML) applications. The project targets a novel domain -- NLP4Test, and this domain requires innovative NLP models. The outcome of this project will include novel techniques, implementations of these techniques, and extensive evaluations on open-source projects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

事实证明，自然语言处理 (NLP) 模型对于各种软件工程任务非常有用，包括代码完成、注释生成和更新、代码审查生成和克隆检测。尽管软件测试在工业中很重要，但使用这些人工智能（AI）模型来开发和维护测试代码的工作却很少，而测试代码是现实世界中软件测试的关键部分。测试代码与常规代码有很多不同之处：（1）测试代码以特定的方式构建，包括设置测试环境和比较预期结果的步骤；（2）测试代码有更丰富的上下文，比如它正在测试的具体方法和代码（待测代码）； (3) 测试代码使用与被测代码不同的代码元素，即具有不同的控制结构； (4) 测试代码有具体的输入值和预期结果； (5) 与常规代码不同，测试代码可以轻松执行。该项目的目标是通过简化测试开发和维护的 NLP 模型来提高软件工程师的生产力（NLP4Test）。具体来说，任务包括测试生成和完成、测试更新（当底层代码更改时）以及跨不同编程语言自动迁移测试。该项目探索测试通用代码库和新兴机器学习 (ML) 应用程序。该项目针对一个新的领域——NLP4Test，该领域需要创新的 NLP 模型。该项目的成果将包括新技术、这些技术的实施以及对开源项目的广泛评估。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。