CHS:Small: Incorporating and Balancing Stakeholder Values in Algorithm Design

CHS:Small：在算法设计中纳入并平衡利益相关者价值观

• 批准号: 2001851
• 负责人: Haiyi Zhu
• 金额: $ 50万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2001851&HistoricalAwards=false
• 关键词: CHS; Small; Incorporating; Balancing; Stakeholder

This project will create a general method for value-sensitive algorithm design and develop tools and techniques to help incorporate the tacit values of stakeholders, balance multiple stakeholders' values, and achieve collective goals in the development of an algorithm. The research community has paid increasing attention to the role of human values in algorithm design and development. For example, fairness-aware machine learning research attempts to translate fairness notions into formal algorithmic constraints and develop algorithms subject to such constraints. Despite the mathematical rigor of these approaches, prior research suggests a disconnect between the current discrimination-aware machine learning research and stakeholders' realities, context, and constraints; this disconnect is likely to undermine practical initiatives. Furthermore, studies have suggested that there are often tensions among a diverse set of values relevant to the design of the algorithm. A new general method will be developed in the context of Redesigning Wikipedia's Objective Revision Evaluation Service (ORES), a machine learning-based service designed to generate real-time predictions on edit quality and article quality, which will benefit vast numbers of people who consume the Wikipedia content either directly or indirectly through other applications.There are four major goals of this research. The first is to articulate and demonstrate a general method for creating algorithmic systems that respect and balance stakeholders' values. The second goal is to create techniques for generating an algorithmic system's value report and explaining the value trade-offs. The third goal to create, deploy, and evaluate social and technical innovations to address fundamental trade-offs between different values. The final goal is to design and implement improvements to ORES, which will improve a wide variety of applications that rely on ORES, and Wikipedia's content and community as a whole. For an example of the kinds of problems that must be solved, quality control algorithms that prioritize efficiency in deleting low quality content incur the risk of undermining the motivation of contributors in peer production communities, particularly new contributors who are still learning how to contribute. To date, however, little work has been conducted to create solutions to address tensions and trade-offs between different values in algorithm design. The research will be performed through multiple studies by stepping through the process for a diverse set of tasks, each of which will allow interaction with multiple stakeholders, who have different (and perhaps conflicting) values.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将创建一种价值敏感算法设计的通用方法，并开发工具和技术，以帮助整合利益相关者的隐性价值，平衡多个利益相关者的价值观，并实现算法开发中的集体目标。 研究界越来越关注人类价值观在算法设计和开发中的作用。例如，具有公平意识的机器学习研究试图将公平概念转化为正式的算法约束，并开发受此类约束的算法。尽管这些方法具有数学严谨性，但先前的研究表明，当前的歧视感知机器学习研究与利益相关者的现实、背景和约束之间存在脱节；这种脱节可能会破坏实际举措。此外，研究表明，与算法设计相关的不同价值观之间经常存在紧张关系。 将在重新设计维基百科的客观修订评估服务（ORES）的背景下开发一种新的通用方法，这是一种基于机器学习的服务，旨在生成编辑质量和文章质量的实时预测，这将使广大消费者受益直接或间接通过其他应用程序获取维基百科内容。这项研究有四个主要目标。 首先是阐明和演示创建尊重和平衡利益相关者价值观的算法系统的通用方法。 第二个目标是创建用于生成算法系统的价值报告并解释价值权衡的技术。第三个目标是创建、部署和评估社会和技术创新，以解决不同价值观之间的基本权衡问题。 最终目标是设计和实施对 ORES 的改进，这将改善依赖 ORES 的各种应用程序以及整个维基百科的内容和社区。 举一个必须解决的问题类型的例子，优先考虑删除低质量内容的效率的质量控制算法会带来削弱同行生产社区中贡献者的积极性的风险，特别是仍在学习如何贡献的新贡献者。然而，迄今为止，几乎没有开展工作来创建解决方案来解决算法设计中不同值之间的紧张关系和权衡。 该研究将通过多项研究进行，逐步完成一系列不同的任务，每项任务都将允许与具有不同（甚至可能冲突）价值观的多个利益相关者进行互动。该奖项反映了 NSF 的法定使命，并被视为值得通过使用基金会的智力优点和更广泛的影响审查标准进行评估来支持。

项目成果

Keeping Designers in the Loop: Communicating Inherent Algorithmic Trade-offs Across Multiple Objectives

让设计师了解情况：沟通跨多个目标的固有算法权衡

• DOI: 10.1145/3357236.3395528
• 发表时间: 2020-07
• 期刊: Proceedings of the 2020 ACM Designing Interactive Systems Conference
• 作者: Yu, Bowen;Yuan, Ye;Terveen, Loren;Wu, Zhiwei Steven;Forlizzi, Jodi;Zhu, Haiyi
• 通讯作者: Zhu, Haiyi

How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions

儿童福利工作者如何减少算法决策中的种族差异

• DOI: 10.1145/3491102.3501831
• 发表时间: 2022-04
• 期刊: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
• 作者: Cheng, Hao;Stapleton, Logan;Kawakami, Anna;Sivaraman, Venkatesh;Cheng, Yanghuidi;Qing, Diana;Perer, Adam;Holstein, Kenneth;Wu, Zhiwei Steven;Zhu, Haiyi
• 通讯作者: Zhu, Haiyi

Join, Stay or Go?: A Closer Look at Members' Life Cycles in Online Health Communities

加入、留下还是离开？：仔细观察在线健康社区成员的生命周期

• DOI: 10.1145/3449245
• 发表时间: 2021-04
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Yao, Zheng;Yang, Diyi;Levine, John M.;Low, Carissa A.;Smith, Tenbroeck;Zhu, Haiyi;Kraut, Robert E.
• 通讯作者: Kraut, Robert E.

Mental health during the COVID-19 pandemic: Impacts of disease, social isolation, and financial stressors

COVID-19 大流行期间的心理健康：疾病、社会隔离和财务压力的影响

• DOI: 10.1371/journal.pone.0277562
• 发表时间: 2022-11
• 期刊: PLOS ONE
• 影响因子: 3.7
• 作者: Kraut, Robert E.;Li, Han;Zhu, Haiyi
• 通讯作者: Zhu, Haiyi

Learning to Become a Volunteer Counselor: Lessons from a Peer-to-Peer Mental Health Community

学习成为一名志愿者咨询师：来自同伴心理健康社区的经验教训

• DOI: 10.1145/3555200
• 发表时间: 2022-11
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Yao, Zheng;Zhu, Haiyi;Kraut, Robert E.
• 通讯作者: Kraut, Robert E.