Using AI to improve our understanding of verbal confidence and to aid decision-making: Eyewitness lineup identification as a model case

使用人工智能提高我们对言语信心的理解并辅助决策：以目击者阵容识别为典型案例

• 批准号: 2241989
• 负责人: Chad Dodson
• 金额: $ 36.7万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2241989&HistoricalAwards=false
• 关键词: Using; AI; improve; our; understanding

From medical treatment to predicting recidivism, artificial intelligence (AI) is playing an increasingly larger role in human decision-making – in large part because AI predictions are superior to human predictions in a variety of different domains. This project uses eyewitness lineup identifications as a model paradigm. One aim of the project uses AI to better understand verbal expressions of confidence about an identification. For example, when an eyewitness states “I’m pretty sure it’s him” about a lineup identification what is the likelihood that the eyewitness’s identification is correct? Another aim of the project examines how best to convey AI output to people so as to improve their predictions about the accuracy of an eyewitness’s identification. The project consists of two sets of experiments. One set uses variations on an eyewitness memory paradigm to examine the predictive value of verbal (e.g., “I’m pretty certain”) and numeric (e.g., “I’m 75% certain”) expressions of confidence. It is largely assumed that verbal confidence statements reflect the same underlying information as numeric confidence ratings. Machine-learning classifiers are used to quantify verbal confidence to explain why verbal confidence is not redundant with numeric confidence but can contribute unique added value in predicting the accuracy of a response. A second set of experiments test two predictions. First, that machine learning estimates about the accuracy of a lineup identification are more resistant than human estimates to the effects of various kinds of contextual information. Second, that a cognitive-forcing method of conveying machine-learning estimates about eyewitness identification accuracy is most effective at improving people’s predictions about eyewitness accuracy, particularly under conditions when people’s intuitions about eyewitness performance are wrong. Overall, this project addresses a need to identify issues in human-algorithm interactions before the spread of AI-assistance to the domain of eyewitness identification.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从医疗到预测累犯，人工智能（AI）在人类决策中起着越来越大的作用 - 在很大程度上是因为AI预测在各种不同领域中都优于人类预测。该项目将目击者阵容标识作为模型范式。该项目的一个目的使用AI更好地理解对身份证明的信心的口头表达。例如，当目击者说“我很确定是他”时，关于阵容的身份证明，目击者的身份是正确的？项目检查的另一个目的是如何最好地将AI输出传达给人们，以提高他们对目击者识别准确性的预测。该项目由两组实验组成。一组在目击者记忆范式上使用变体来检查语言（例如“我很确定”）和数字（例如，“我是75％的某些确定”）表达信心的预测值。在很大程度上假定语言置信度语言反映了与数字置信度等级相同的基础信息。机器学习分类器用于量化言语置信度，以解释为什么言语置信度不是具有数值置信度的冗余，而是可以在预测响应的准确性方面贡献独特的附加值。第二组实验测试了两个预测。首先，该机器学习估计阵容识别的准确性比人类对各种上下文信息影响的估计更具抵抗力。其次，传达有关目击者识别准确性的机器学习估计值的认知方法最有效地改善人们对目击者准确性的预测，尤其是在人们对目击者表现的直觉是错误的情况下。总体而言，该项目解决了需要在AI辅助对目击者识别领域的传播之前确定人类算法相互作用中的问题。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响标准通过评估来评估的支持。