EAGER: Exploring the Feasibility of Software Testing Techniques to Evaluate Fairness Algorithms in Software Systems

EAGER：探索软件测试技术评估软件系统公平算法的可行性

• 批准号: 1744471
• 负责人: Yuriy Brun
• 金额: $ 13.12万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1744471&HistoricalAwards=false
• 关键词: EAGER; Exploring; Feasibility; Software; Testing

Today, software is making more automated decisions with societal impact. For example, software determines who gets a loan or gets hired, computes risk-assessment scores that help decide who goes to jail and who is set free, and aids in diagnosing and treating medical patients. The increased role of software in such decisions makes software fairness a critical property. As more societal functions operate in cyberspace, the importance of software fairness increases. This project evaluates the feasibility of using software testing technology to identify behaviors whose outputs are more favorable for certain inputs. Using testing in such a manner is aimed at capturing causal relationships between characteristics of the software inputs and the software behavior. The approach is novel compared to typical machine learning classification techniques that analyze data but do not test the behavior of application software. The central idea is to identify causal relationships between software inputs and the way the software behaves, e.g., its outputs. Software testing enables conducting causal experiments consisting of running the software with nearly identical inputs that vary only in a key characteristic under test. Variations in that characteristic that affect behavior provide evidence of a causal relationship. Measuring such causal relationships requires test suites that focus on small variability in a key set of characteristics under test, while existing testing techniques focus on large variability that leads to greater coverage. As a result, existing techniques are ill-suited for measuring causal relationships and new technology is necessary. The result is the ability to test software for new properties for which no testing procedures existed.

如今，软件正在做出更加自动化的决策，并产生社会影响。例如，软件确定谁获得贷款或被雇用，计算风险评估分数以帮助决定谁入狱和谁被释放，并帮助诊断和治疗病人。软件在此类决策中的作用日益增强，使得软件公平性成为一个关键属性。随着越来越多的社会功能在网络空间中运作，软件公平的重要性也随之增加。该项目评估使用软件测试技术来识别其输出对某些输入更有利的行为的可行性。以这种方式使用测试的目的是捕获软件输入特征和软件行为之间的因果关系。 与分析数据但不测试应用软件行为的典型机器学习分类技术相比，该方法是新颖的。中心思想是识别软件输入与软件行为方式（例如其输出）之间的因果关系。软件测试可以进行因果实验，包括使用几乎相同的输入运行软件，这些输入仅在测试的关键特性上有所不同。影响行为的特征的变化提供了因果关系的证据。测量这种因果关系需要测试套件专注于被测试的一组关键特征中的小变化，而现有的测试技术则专注于导致更大覆盖范围的大变化。因此，现有技术不适合测量因果关系，因此需要新技术。其结果是能够测试软件的新属性，而这些新属性不存在测试程序。

项目成果

Tortoise: Interactive system configuration repair

Tortoise：交互式系统配置修复

• DOI: 10.1109/ase.2017.8115673
• 发表时间: 2017
• 期刊: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE
• 作者: Weiss, Aaron;Guha, Arjun;Brun, Yuriy
• 通讯作者: Brun, Yuriy

RC-Index: Diversifying Answers to Range Queries

• DOI: 10.14778/3192965.3192969
• 发表时间: 2018-03
• 期刊: Proc. VLDB Endow.
• 作者: Yue Wang;A. Meliou;G. Miklau

Software Fairness

• DOI: 10.1145/3236024.3264838
• 发表时间: 2018-01-01
• 期刊: ESEC/FSE'18: PROCEEDINGS OF THE 2018 26TH ACM JOINT MEETING ON EUROPEAN SOFTWARE ENGINEERING CONFERENCE AND SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING
• 作者: Brun, Yuriy;Meliou, Alexandra
• 通讯作者: Meliou, Alexandra

Themis: Automatically testing software for discrimination

Themis：自动测试软件的歧视性

• DOI: 10.1145/3236024.3264590
• 发表时间: 2018
• 期刊: Proceedings of the Demonstrations Track at the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE
• 作者: Angell, Rico;Johnson, Brittany;Brun, Yuriy;Meliou, Alexandra
• 通讯作者: Meliou, Alexandra

Fairness Testing: Testing Software for Discrimination

• DOI: 10.1145/3106237.3106277
• 发表时间: 2017-01-01
• 期刊: ESEC/FSE 2017: PROCEEDINGS OF THE 2017 11TH JOINT MEETING ON FOUNDATIONS OF SOFTWARE ENGINEERING
• 作者: Galhotra, Sainyam;Brun, Yuriy;Meliou, Alexandra
• 通讯作者: Meliou, Alexandra