Doctoral Dissertation Research in DRMS: Connecting Artificial Intelligence Literacy and Human-AI Decision Making Outcomes in Organizational Hiring

DRMS 博士论文研究：将人工智能素养与组织招聘中的人类人工智能决策成果联系起来

• 批准号: 2117860
• 负责人: Keri Stephens
• 金额: $ 2.8万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2117860&HistoricalAwards=false
• 关键词: Doctoral; Dissertation; Research; DRMS; Connecting

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Organizations are increasingly incorporating artificial intelligence (AI)-based technologies into decision-making processes. For example, hiring teams may use AI-based tools that analyze application data, such as resumes and interview recordings, to provide hiring recommendations. AI-based algorithms can process large amounts of data and generate recommendations that inform decisions that used to be made solely by human decision-makers. Despite the widespread use of AI-based technologies, everyday non-expert users of these technologies may not have sufficient knowledge about AI, or AI literacy, to make fair decisions using AI-based recommendations. This Doctoral Dissertation Research Improvement Grant (DDRIG) examines whether receiving recommendations from an AI-based source impacts outcomes when decision-makers vary in their AI literacy. This project has two primary purposes: 1) to develop a measure for people's AI literacy within the context of decision-making, and 2) to test this measure in a hiring scenario experiment. This research supports NSF’s mission to promote the progress of science by contributing tools that support further research on human-AI interaction, a subject becoming more relevant as organizations continue to introduce new AI-based technologies into the workplace. This work helps people, regardless of technical background, become better informed and thoughtful human-AI decision-makers. The findings from this research advance a validated measure for AI literacy that researchers, individuals, and organizations can use to measure individuals' general understanding of AI-based technologies and identify potential gaps in knowledge that can impact how they use AI-based information to make decisions. This research also informs the development of educational resources that help job seekers navigate the AI-based hiring process.The research entails two phases. Phase I involves a scale development effort that uses past research and pilot interview data to develop and test scale items for an AI literacy measure. To determine whether people's AI literacy plays a significant role in how people use AI-based information to make hiring recommendations, phase II involves a hiring scenario experiment. In the experiment, participants evaluate a mock job application and use input from a secondary evaluation to decide whether the applicant should move forward in the hiring process, and participants' AI literacy (as measured by the scale developed in phase I) is used as a control variable. The results from this investigation may help various stakeholders better understand how people's understanding of AI influences decision-making outcomes. The findings have the potential to contribute to work on AI training and education at large. Furthermore, the experiment is the first application of the AI literacy measure that catalyzes future research exploring the relationship between AI literacy and human-AI decision-making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项是根据2021年《美国救援计划法》的全部或部分资助（公共法117-2）。组织越来越多地将基于人工智能（AI）的技术增加到决策过程。例如，招聘团队可以使用基于AI的工具来分析应用程序数据，例如简历和采访记录，以提供招聘建议。基于AI的算法可以处理大量数据并生成建议，以告知人们以前仅由人类决策者制定的决策。尽管使用了基于AI的技术的广泛使用，但这些技术的非专家用户可能没有足够的知识来使用基于AI的建议来做出公正的决定。这项博士学位论文研究改进赠款（DDRIG）检查是否会从基于AI的来源获得建议会影响决策者的AI识字率各不相同时会影响结果。该项目有两个主要目的：1）在决策背景下制定人们的AI素养措施，以及2）在招聘场景实验中测试此措施。这项研究支持NSF通过贡献有关人类互动的进一步研究来促进科学进步的使命，随着组织继续将新的基于AI的技术引入工作场所，这一主题变得越来越重要。这项工作可以帮助人们，无论技术背景如何，都变得更加知名和周到的人类决策者。这项研究的发现提高了对AI素养的经过验证的测量，研究人员，个人和组织可以用来衡量个人对基于AI的技术的一般理解，并确定知识的潜在差距，从而影响他们使用基于AI的信息来做出决策的方式。这项研究还为教育资源的发展提供了信息，这些教育资源可帮助求职者浏览基于AI的招聘过程。研究需要两个阶段。第一阶段涉及一项规模开发工作，该工作使用过去的研究和试点访谈数据来开发和测试规模项目以进行AI扫盲度量。为了确定人们的AI素养在人们如何使用基于AI的信息来提出建议方面是否起着重要作用，第二阶段涉及招聘场景实验。在实验中，参与者评估了模拟工作应用程序，并使用次级评估的输入来决定该应用程序是否应在招聘过程中前进，并且参与者的AI素养（由I阶段中开发的量表衡量）用作控制变量。这项投资的结果可能有助于各种利益相关者更好地了解人们对AI的理解如何影响决策结果。这些发现有可能为整个AI培训和教育的工作做出贡献。此外，该实验是AI素养测量的首次应用，该测量催化未来的研究探讨了AI扫盲与人类AI决策之间的关系。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛影响的审查标准通过评估来获得的支持。