CRII: SHF: Machine-Learning-Based Test Effectiveness Prediction

CRII：SHF：基于机器学习的测试有效性预测

• 批准号: 1566589
• 负责人: Lingming Zhang
• 金额: $ 17.42万
• 依托单位: University of Texas at Dallas
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-15 至 2019-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1566589&HistoricalAwards=false
• 关键词: CRII; SHF; Machine; Learning; Based

Test effectiveness, which indicates the capability of tests in detecting potential software bugs, is crucial for software testing. More effective tests can detect more potential bugs and thus help prevent economic loss or even physical damage caused by software bugs. Therefore, a huge body of research efforts have been dedicated to test effectiveness evaluation during the past decades. Recently, mutation testing, a powerful methodology that computes the detection rate of artificially injected bugs to measure test effectiveness, is drawing more and more attention from both the academia and industry. Various studies have shown that artificial bugs generated by mutation testing are close to real bugs, demonstrating mutation testing effectiveness in test effectiveness evaluation. However, a major obstacle for mutation testing is the efficiency problem ? mutation testing requires the execution of each artificial buggy version (i.e., mutant) to check whether the test suite can detect that bug, and which is extremely time consuming. Therefore, a light-weight but precise technique for measuring test effectiveness is highly desirable.The approach is to automatically extract test effectiveness information (e.g., mutation testing results) from various open-source projects to directly predict the test effectiveness of the current project without any mutant execution. More specifically, the PI proposes to design a general classification framework based on a suite of static and dynamic features collected according to the PIE theory of fault detection. Furthermore, this research will explore judicious applications of advanced program analysis, machine learning, and software mining techniques for more powerful feature collection, more active learning, as well as more comprehensive training data preparation. The proposed approach will result in efficient but precise test effectiveness evaluation for projects developed using various programming languages and test paradigms, which is crucial for high-quality software. Furthermore, the training of the classification models will require to collect various basic testing, analysis, and mining information from a huge number of open-source projects, and thus may also benefit a large variety of software testing/analysis/mining techniques that explore open-source software repositories.

测试有效性表明测试在检测潜在软件错误中的能力，对于软件测试至关重要。更有效的测试可以检测到更多的潜在错误，从而有助于防止软件错误造成的经济损失甚至物理损害。因此，在过去的几十年中，已经致力于测试有效性评估的大量研究工作。最近，突变测试是一种强大的方法，它计算人为注入的错误以衡量测试有效性的检测率，它引起了学术界和行业的越来越多的关注。各种研究表明，突变测试产生的人造错误接近实际错误，证明了突变测试在测试有效性评估中的有效性。但是，突变测试的主要障碍是效率问题吗？突变测试需要执行每个人造越野车版本（即突变体），以检查测试套件是否可以检测到该错误，并且非常耗时。因此，高度重量但精确的技术是高度可取的。该方法是从各个开源项目中自动提取测试有效性信息（例如，突变测试结果），以直接预测当前项目的测试有效性而无需任何突变执行。更具体地说，PI建议根据故障检测的PIE理论来设计基于一系列静态和动态特征的通用分类框架。此外，这项研究将探讨高级程序分析，机器学习和软件挖掘技术的明智应用，以收集更强大的功能，更积极的学习以及更全面的培训数据准备。提出的方法将对使用各种编程语言和测试范式开发的项目进行有效但精确的测试有效性评估，这对于高质量的软件至关重要。此外，分类模型的培训将需要从大量开源项目中收集各种基本测试，分析和采矿信息，因此也可能使大量软件测试/分析/采矿技术受益，以探索开源软件存储库。

项目成果

Inferring Program Transformations From Singular Examples via Big Code

• DOI: 10.1109/ase.2019.00033
• 发表时间: 2019-11
• 期刊: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)
• 作者: Jiajun Jiang;Luyao Ren;Yingfei Xiong;Lingming Zhang