CAREER: Participatory Design Methods for Algorithmic Systems

职业：算法系统的参与式设计方法

基本信息

批准号：
1844901
负责人：
Eric Baumer
金额：
$ 54.99万
依托单位：
Lehigh University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-08-01 至 2025-07-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1844901&HistoricalAwards=false
关键词：
CAREER Participatory Design Methods Algorithmic

项目摘要

This research will develop methods for applying participatory design to the underlying components of algorithmic systems. Such systems incorporate increasingly complex algorithms from machine learning (ML), natural language processing (NLP), and related areas into user interactions, intending to achieve great benefits. However, cases of egregious errors, algorithmic bias, and other issues have revealed the shortcomings of relying primarily on quantitative performance metrics to inform design. A potential solution is to incorporate users into the design process. Although human-computer interaction research has developed numerous methods for user-centered design, many such approaches focus primarily at the interface level. This focus becomes problematic when a system's functionality is increasingly determined by ML models and algorithms. Furthermore, designing only for users of algorithmic systems can overlook other important relationships, such as the people whose data are being analyzed or those who may interpret the results. To address these issues, this research will develop participatory methods for human-centered design of algorithmic systems. These methods will be developed and tested by working closely with two non-profit organizations that already engage in data-intensive work but currently make limited use of algorithmic systems: AEquitas, which conducts legal analysis, and ProPublica, an investigative journalism newsroom. Unique challenges emerge when attempting to incorporate different people's relationships with a system into the design process. Existing participatory methods often use visual elements or manipulatives to represent interface components. The abstract mathematical formalisms of algorithmic systems, though, do not always lend themselves to such visual representations. This research will develop novel participatory design techniques to establish common ground between domain experts, who are less familiar with ML or NLP, and researchers, who are unfamiliar with the application domain. Doing so can leverage diverse participants' expertise and interpretations, thereby improving the fit between computational systems and existing practices. The participatory methods directly address underlying technical components, from feature selection, to model construction, to performance evaluation, to result interpretation. Furthermore, these methods will align those underlying technical aspects with current practices and lay understandings, increase the chance of catching and rectifying unanticipated egregious errors before they become problematic, and ensure the results are presented in a transparent and accountable manner. Finally, this process will inform the development of modules for classroom instruction, paired across STEM and social science courses, on how to incorporate human-centered concerns into designing algorithmic systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项研究将开发用于将参与设计应用于算法系统的基本组件的方法。此类系统将机器学习（ML），自然语言处理（NLP）及相关领域的越来越复杂的算法纳入用户互动中，以实现巨大的好处。但是，严重错误，算法偏见和其他问题的案例揭示了主要依赖定量性能指标来告知设计的缺点。潜在的解决方案是将用户纳入设计过程。尽管人类计算机的互动研究开发了许多以用户为中心的设计的方法，但许多这样的方法主要集中在接口级别上。当系统的功能越来越多地由ML模型和算法确定时，该重点就会成为问题。此外，仅针对算法系统的用户设计可以忽略其他重要关系，例如正在分析数据的人或可以解释结果的人。为了解决这些问题，这项研究将开发以人为中心的算法系统设计的参与式方法。这些方法将通过与已经从事数据密集型工作但目前对算法系统的有限使用的非营利组织紧密合作来开发和测试，而进行法律分析的Aequitas和调查新闻室ProPublica。当试图将不同的人的关系与系统融合到设计过程中时，出现了独特的挑战。现有的参与方法通常使用视觉元素或操纵词来表示界面组件。不过，算法系统的抽象数学形式主义并不总是能够以这种视觉表示。这项研究将开发新颖的参与设计技术，以在不熟悉ML或NLP的领域专家和不熟悉应用领域的研究人员之间建立共同点。这样做可以利用不同参与者的专业知识和解释，从而改善计算系统与现有实践之间的拟合度。参与式方法直接解决了从特征选择到模型构造，到绩效评估，再到结果解释的基本技术组成部分。此外，这些方法将使那些潜在的技术方面与当前的实践保持一致，并提出理解，增加在问题之前捕获和纠正意外的严重错误的机会，并确保以透明且负责任的方式提出结果。最后，此过程将为课堂教学模块的发展提供介绍，包括STEM和社会科学课程，如何将以人为中心的关注纳入设计算法系统中。该奖项反映了NSF的法定任务，并认为通过基金会的知识和更广泛的影响，可以通过评估来进行评估，以审查Criteria。

项目成果

期刊论文数量（4）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Topicalizer: reframing core concepts in machine learning visualization by co-designing for interpretivist scholarship

Topicalizer：通过解释主义学术的共同设计重新构建机器学习可视化的核心概念

DOI：
10.1080/07370024.2020.1734460
发表时间：
2020
期刊：
Human–Computer Interaction
影响因子：
0
作者：
Baumer, Eric P.;Siedel, Drew;McDonnell, Lena;Zhong, Jiayun;Sittikul, Patricia;McGee, Micki
通讯作者：
McGee, Micki

Where Do Stories Come From? Examining the Exploration Process in Investigative Data Journalism

故事从何而来？