Doctoral Dissertation Research in DRMS: Connecting Artificial Intelligence Literacy and Human-AI Decision Making Outcomes in Organizational Hiring

DRMS 博士论文研究：将人工智能素养与组织招聘中的人类人工智能决策成果联系起来

• 批准号: 2117860
• 负责人: Keri Stephens
• 金额: $ 2.8万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2117860&HistoricalAwards=false
• 关键词: Doctoral; Dissertation; Research; DRMS; Connecting

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Organizations are increasingly incorporating artificial intelligence (AI)-based technologies into decision-making processes. For example, hiring teams may use AI-based tools that analyze application data, such as resumes and interview recordings, to provide hiring recommendations. AI-based algorithms can process large amounts of data and generate recommendations that inform decisions that used to be made solely by human decision-makers. Despite the widespread use of AI-based technologies, everyday non-expert users of these technologies may not have sufficient knowledge about AI, or AI literacy, to make fair decisions using AI-based recommendations. This Doctoral Dissertation Research Improvement Grant (DDRIG) examines whether receiving recommendations from an AI-based source impacts outcomes when decision-makers vary in their AI literacy. This project has two primary purposes: 1) to develop a measure for people's AI literacy within the context of decision-making, and 2) to test this measure in a hiring scenario experiment. This research supports NSF’s mission to promote the progress of science by contributing tools that support further research on human-AI interaction, a subject becoming more relevant as organizations continue to introduce new AI-based technologies into the workplace. This work helps people, regardless of technical background, become better informed and thoughtful human-AI decision-makers. The findings from this research advance a validated measure for AI literacy that researchers, individuals, and organizations can use to measure individuals' general understanding of AI-based technologies and identify potential gaps in knowledge that can impact how they use AI-based information to make decisions. This research also informs the development of educational resources that help job seekers navigate the AI-based hiring process.The research entails two phases. Phase I involves a scale development effort that uses past research and pilot interview data to develop and test scale items for an AI literacy measure. To determine whether people's AI literacy plays a significant role in how people use AI-based information to make hiring recommendations, phase II involves a hiring scenario experiment. In the experiment, participants evaluate a mock job application and use input from a secondary evaluation to decide whether the applicant should move forward in the hiring process, and participants' AI literacy (as measured by the scale developed in phase I) is used as a control variable. The results from this investigation may help various stakeholders better understand how people's understanding of AI influences decision-making outcomes. The findings have the potential to contribute to work on AI training and education at large. Furthermore, the experiment is the first application of the AI literacy measure that catalyzes future research exploring the relationship between AI literacy and human-AI decision-making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项的全部或部分资金根据《2021 年美国救援计划法案》（公法 117-2）提供。组织越来越多地将基于人工智能 (AI) 的技术纳入决策流程，例如，招聘团队可能会使用这些技术。基于人工智能的工具可以分析简历和面试录音等申请数据，以提供基于人工智能的算法，可以处理大量数据并生成建议，为过去仅由人类决策者做出的决策提供信息。广泛使用对于基于人工智能的技术，这些技术的日常非专家用户可能没有足够的人工智能知识或人工智能素养，无法使用基于人工智能的建议做出公平的决策。这项博士论文研究改进补助金 (DDRIG) 检查是否收到来自人工智能的建议。当决策者的人工智能素养不同时，基于人工智能的来源会影响结果。该项目有两个主要目的：1) 在决策背景下制定一种衡量人们人工智能素养的衡量标准，2) 测试该衡量标准。招聘场景实验。这项研究支持 NSF 的使命，即通过提供支持人机交互进一步研究的工具来促进科学进步，随着组织不断将基于人工智能的新技术引入工作场所，这一主题变得越来越重要。这项工作可以帮助人们，无论技术如何。这项研究的结果提出了一种经过验证的人工智能素养衡量标准，研究人员、个人和组织可以使用它来衡量个人对基于人工智能的技术的总体理解并找出潜在的差距。知识可以影响他们如何使用基于人工智能的信息来制作这项研究还为教育资源的开发提供了信息，帮助求职者驾驭基于人工智能的招聘流程。该研究分为两个阶段，包括使用过去的研究和试点面试数据来开发和测试规模项目。为了确定人们的人工智能素养是否在人们如何使用基于人工智能的信息提出招聘建议方面发挥着重要作用，第二阶段涉及招聘场景实验，在实验中，参与者评估模拟工作申请并使用输入。通过二次评估来决定申请人是否应该继续前进招聘过程和参与者的人工智能素养（按照第一阶段制定的量表来衡量）被用作控制变量。这项调查的结果可以帮助各个利益相关者更好地了解人们对人工智能的理解如何影响决策结果。有潜力为人工智能培训和教育工作做出贡献。此外，该实验是人工智能素养测量的首次应用，可促进未来探索人工智能素养与人类人工智能决策之间关系的研究。反映了 NSF 的法定使命并被认为值得通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。