CRII: CHS: Experimental Studies of Human Trust in Machine Learning

CRII：CHS：机器学习中人类信任的实验研究

基本信息

批准号：
1850335
负责人：
Ming Yin
金额：
$ 17.5万
依托单位：
Purdue University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-04-15 至 2023-03-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1850335&HistoricalAwards=false
关键词：
CRII CHS Experimental Studies Human

项目摘要

Machine learning (ML) has been applied to various domains including finance, healthcare, urban operations, and targeted news and advertising. They achieve great success uncovering insights from massive data and advancing decision making. Despite widespread applications, scientific understanding of peoples' trust or lack of trust in ML approaches is lacking, while the success of human-machine collaborations requires a deeper understanding of trust. This project advances the understanding of lay people's trust in ML through large-scale randomized human-subject experiments. The project will result in theoretical insights on the formation and maintenance of trust between humans and ML systems, and practical insights for designing systems acceptable to people.Leveraging existing theoretical models of trust in automation, this project will answer three fundamental questions related to lay people's trust in machine learning. How does the performance of an ML system (correctness, reliability, and predictability) affect people's trust in it? How does its interpretability affect people's trust in it? How do the performance and interpretability interact with each other to influence trust? The project will:(1) advance theoretical and empirical understanding about the development and maintenance of trust in ML systems (and compare with trust in conventional automated systems), (2) provide design guidelines that help instill trust in ML systems, and (3) develop an experimental framework for evaluating and benchmarking trustworthiness of ML systems in a systematic manner.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习（ML）已应用于金融、医疗、城市运营、定向新闻和广告等各个领域。他们在从海量数据中揭示见解并推进决策方面取得了巨大成功。尽管应用广泛，但人们对机器学习方法的信任或缺乏信任缺乏科学理解，而人机协作的成功需要对信任有更深入的理解。该项目通过大规模随机人体实验加深了人们对机器学习的信任。该项目将产生关于人类与机器学习系统之间信任的形成和维护的理论见解，以及设计人们可以接受的系统的实践见解。利用现有的自动化信任理论模型，该项目将回答与外行人相关的三个基本问题对机器学习的信任。机器学习系统的性能（正确性、可靠性和可预测性）如何影响人们对其的信任？它的可解释性如何影响人们对它的信任？性能和可解释性如何相互作用来影响信任？该项目将：(1) 增进对 ML 系统信任的开发和维护的理论和实证理解（并与传统自动化系统中的信任进行比较），(2) 提供有助于灌输对 ML 系统信任的设计指南，以及 (3 ）开发一个实验框架，以系统的方式评估和基准化机器学习系统的可信度。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（12）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

The Effects of AI-based Credibility Indicators on the Detection and Spread of Misinformation under Social Influence

基于人工智能的可信度指标对社会影响下错误信息检测和传播的影响

DOI：
10.1145/3555562
发表时间：
2022-11
期刊：
Proceedings of the ACM on Human-Computer Interaction
影响因子：
0
作者：
Lu, Zhuoran;Li, Patrick;Wang, Weilong;Yin, Ming
通讯作者：
Yin, Ming

Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making

你会接受人工智能的推荐吗？

DOI：
10.1145/3485447.3512240
发表时间：
2022-04
期刊：
Proceedings of the 2022 ACM Web Conference (WWW
影响因子：
0
作者：
Wang, Xinru;Lu, Zhuoran;Yin, Ming
通讯作者：
Yin, Ming

Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks

当绩效反馈有限时，人类对机器学习模型的依赖：启发法和风险

DOI：
10.1145/3411764.3445562
发表时间：
2021-05-06
期刊：
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
影响因子：
0
作者：
Zhuoran Lu;Ming Yin
通讯作者：
Ming Yin

When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models

当信心满足准确性：探索多个绩效指标对机器学习模型信任的影响

DOI：
10.1145/3491102.3501967
发表时间：
2022-04-29
期刊：
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
影响因子：
0
作者：
Amy Rechkemmer;Ming Yin
通讯作者：
Ming Yin

You’d Better Stop! Understanding Human Reliance on Machine Learning Models under Covariate Shift

你最好停下来！