InfoTestSS: Information theory and Test Suite Selection

InfoTestSS：信息论和测试套件选择

• 批准号: EP/P005888/1
• 负责人: David Clark
• 金额: $ 57.08万
• 依托单位: University College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2017
• 资助国家: 英国
• 起止时间: 2017 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FP005888%2F1
• 关键词: InfoTestSS; Information; theory; Test; Suite

Software testing is an important part of the software development process but typically is manual, expensive, and error prone. This has led to significant interest in automated test generation (and execution) algorithms, with these having the potential to lead to cheaper, higher-quality software. Despite the interest in automating parts of testing, there are still significant challenges, with auto-testing being mentioned as an EPSRC priority within Software Engineering.This project will build on initial work by the PIs that has demonstrated that an important aspect of testing can be represented in terms of Quantified Information Flow. Specifically, the PIs previously looked at Failed Error Propagation (FEP), which is sometimes called coincidental correctness. In FEP, a test execution goes through a faulty part of the software, this leads to what would be regarded as a corrupted program state (i.e. the fault has an effect) but ultimately the output is correct. Although studies have shown that FEP can significantly reduce test effectiveness, there is a lack of practical techniques that address FEP. The observation made by the PIs is that FEP corresponds to a failure for information to flow from the fault in the software to output: information is lost through different values for the program state (correct and faulty values) being mapped to the same output.The PIs have shown how FEP can be represented in terms of an information theoretic notion: Quantified Information Flow (QIF). The results of experiments were highly promising, with there being a rank correlation of over 0.95 between the frequency with which FEP was observed in software and a QIF-based metric. This remarkably strong result opens up the possibility of devising techniques that generate test cases that are less likely to suffer from FEP. In addition, we believe that it is possible to represent other important testing concepts using information theory, specifically: the 'feasibility' of a path (we do not want test automation to waste effort in trying to trigger infeasible paths), the diversity of a test suite (evidence suggests that diverse test suites are effective), and also the effectiveness of probes/oracles added to the code.This project will develop new methods, based on information theory, for reasoning about the above factors (FEP, feasibility, diversity, and oracles). In doing so it will develop information theoretic measures that can help test automation to overcome the associated issues. It will also develop methods for estimating these measures, integrate these estimates into automated test generation, and evaluate the results on open source software and software provided by our industrial partners. The outcome will be a new theory for software testing, based on information theory, and a set of techniques that use this theory to make software testing more efficient and effective.

软件测试是软件开发过程的重要组成部分，但通常是手动的、昂贵的且容易出错。这引起了人们对自动测试生成（和执行）算法的极大兴趣，这些算法有可能带来更便宜、更高质量的软件。尽管人们对自动化部分测试感兴趣，但仍然存在重大挑战，自动测试被认为是软件工程中 EPSRC 的优先事项。该项目将建立在 PI 的初步工作基础上，这些工作已证明测试的一个重要方面可以是以量化信息流的形式表示。具体来说，PI 之前关注的是失败错误传播 (FEP)，有时称为巧合正确性。在 FEP 中，测试执行会经历软件的错误部分，这会导致被视为损坏的程序状态（即错误产生影响），但最终输出是正确的。尽管研究表明 FEP 会显着降低测试有效性，但缺乏解决 FEP 的实用技术。 PI 的观察结果是，FEP 对应于信息从软件中的故障流向输出的故障：通过将程序状态的不同值（正确值和错误值）映射到同一输出，信息会丢失。 PI 已经展示了如何用信息理论概念来表示 FEP：量化信息流 (QIF)。实验结果非常有希望，在软件中观察到 FEP 的频率与基于 QIF 的指标之间存在超过 0.95 的等级相关性。这一非常有力的结果为设计生成不太可能遭受 FEP 的测试用例的技术提供了可能性。此外，我们相信可以使用信息论来表示其他重要的测试概念，特别是：路径的“可行性”（我们不希望测试自动化浪费精力尝试触发不可行的路径）、路径的多样性测试套件（证据表明不同的测试套件是有效的），以及添加到代码中的探针/预言机的有效性。该项目将开发基于信息论的新方法，用于推理上述因素（FEP、可行性、多样性） ， 和神谕）。在此过程中，它将开发信息理论措施，帮助测试自动化克服相关问题。它还将开发估计这些措施的方法，将这些估计集成到自动测试生成中，并评估开源软件和我们的工业合作伙伴提供的软件的结果。其成果将是一种基于信息论的新软件测试理论，以及一系列使用该理论使软件测试更加高效和有效的技术。

项目成果

Search-Based Software Engineering - 11th International Symposium, SSBSE 2019, Tallinn, Estonia, August 31 - September 1, 2019, Proceedings

基于搜索的软件工程 - 第 11 届国际研讨会，SSBSE 2019，爱沙尼亚塔林，2019 年 8 月 31 日至 9 月 1 日，会议记录

• DOI: http://dx.10.1007/978-3-030-27455-9_13
• 发表时间: 2019
• 作者: Bruce D

Fitness first and fatherless crossover

健身第一与无父交叉

• DOI: http://dx.10.1145/3449726.3459437
• 发表时间: 2021
• 作者: Langdon W

Oracle problem in software testing

Oracle软件测试中的问题

• DOI: 10.1145/3092703.3098235
• 发表时间: 2017-07-10
• 期刊: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis
• 作者: Gunel Jahangirova

Genetic Programming - 24th European Conference, EuroGP 2021, Held as Part of EvoStar 2021, Virtual Event, April 7-9, 2021, Proceedings

基因编程 - 第 24 届欧洲会议，EuroGP 2021，作为 EvoStar 2021 的一部分举行，虚拟活动，2021 年 4 月 7-9 日，会议记录

• DOI: http://dx.10.1007/978-3-030-72812-0_15
• 发表时间: 2021
• 作者: Langdon W

Normalised Squeeziness and Failed Error Propagation

归一化挤压性和失败的错误传播

• DOI: http://dx.10.1016/j.ipl.2019.04.001
• 发表时间: 2019
• 期刊: Information Processing Letters
• 影响因子: 0.5
• 作者: Clark D