SHF: MEDIUM: Achieving Software Reliability without True Test Oracles

SHF：中：在没有真正的测试预言机的情况下实现软件可靠性

基本信息

批准号：
1161079
负责人：
Gail Kaiser
金额：
$ 89.46万
依托单位：
Columbia University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2012
资助国家：
美国
起止时间：
2012-09-01 至 2016-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1161079&HistoricalAwards=false
关键词：
SHF MEDIUM Achieving Software Reliability

项目摘要

Conventional software testing checks whether each output is correct for the set of test inputs. But for some software, it is not known what the correct output should be for some inputs -- yet it is still important to detect coding errors in that software, so they can be fixed. This dilemma arises frequently for machine learning, simulation and optimization applications, often "Programs which were written in order to determine the answer in the first place. There would be no need to write such programs, if the correct answer were known." As these kinds of applications are frequently used in public infrastructure and biomedical research (domains targeted in this research), it is critical to detect and fix errors before a calamity occurs. Fortunately, many such applications reflect 'metamorphic properties' that define a relationship between pairs of inputs and outputs, such that for any previous input i with its already known output o, one can easily derive a test input i' and predict the expected output o'. If the actual output o'' is different from o', then there must be an error in the code. This project investigates methodology for determining the metamorphic properties of software and for devising good test cases from which the secondary tests can be derived. The project extends the inputs/outputs considered in previous work on metamorphic testing to focus on application state, before and after, rather than just functional parameters and results. The research also extends the pairwise relations implied by metamorphic properties to 'semantic similarity' for nondeterministic applications, applied to profiles from numerous executions, since an exact relation cannot be expected to hold for a single pair of test executions. These extensions enable treatment of more sophisticated properties that preliminary experiments have shown to reveal defects that were not detected otherwise.

传统的软件测试检查每个输出对于测试输入集是否正确。但对于某些软件来说，不知道某些输入的正确输出应该是什么 - 但检测该软件中的编码错误仍然很重要，以便可以修复它们。这种困境在机器学习、模拟和优化应用中经常出现，通常是“为了首先确定答案而编写的程序。如果知道正确的答案，就不需要编写这样的程序。”由于此类应用程序经常用于公共基础设施和生物医学研究（本研究的目标领域），因此在灾难发生之前检测并修复错误至关重要。幸运的是，许多此类应用程序反映了定义输入和输出对之间关系的“变形属性”，这样对于任何先前的输入 i 及其已知的输出 o，人们可以轻松导出测试输入 i' 并预测预期输出 o '。如果实际输出 o'' 与 o' 不同，那么代码中一定有错误。该项目研究了确定软件变形属性和设计可从中导出二次测试的良好测试用例的方法。该项目扩展了之前变质测试工作中考虑的输入/输出，以关注之前和之后的应用程序状态，而不仅仅是功能参数和结果。该研究还将变质属性所暗示的成对关系扩展到非确定性应用程序的“语义相似性”，应用于来自多次执行的配置文件，因为不能期望精确的关系适用于单对测试执行。这些扩展能够处理更复杂的属性，初步实验表明这些属性可以揭示以其他方式无法检测到的缺陷。

项目成果

期刊论文数量（10）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Vroom: Faster Build Processes for Java

Vroom：更快的 Java 构建过程

DOI：
10.1109/ms.2015.32
发表时间：
2015
期刊：
IEEE Software
影响因子：
3.3
作者：
Bell, Jonathan;Melski, Eric;Dattatreya, Mohan;Kaiser, Gail E.
通讯作者：
Kaiser, Gail E.

Code relatives: detecting similarly behaving software

DOI：
10.1145/2950290.2950321
发表时间：
2016-11
期刊：
Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering
影响因子：
0
作者：
Fang-Hsiang Su;Jonathan Bell;Kenneth Harvey;S. Sethumadhavan;G. Kaiser;Tony Jebara
通讯作者：
Fang-Hsiang Su;Jonathan Bell;Kenneth Harvey;S. Sethumadhavan;G. Kaiser;Tony Jebara

Metamorphic Runtime Checking of Applications without Test Oracles