AI-DCL: Collaborative Research: EAGER: Understanding and Alleviating Potential Biases in Large Scale Employee Selection Systems: The Case of Automated Video Interviews

AI-DCL：协作研究：EAGER：理解和减轻大规模员工选拔系统中的潜在偏见：自动视频面试的案例

• 批准号: 1921087
• 负责人: Sidney D'Mello
• 金额: $ 14.5万
• 依托单位: University of Colorado at Boulder
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-15 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1921087&HistoricalAwards=false
• 关键词: AI; DCL; Collaborative; Research; EAGER

The goal of this project is to use machine learning to understand and mitigate bias in interviewer evaluations. The researchers will do so by examining gender differences in expressed behavior during interviews; they will focus on behaviors that can lead to different interviewer evaluations. More specifically, they will use unsupervised video interviews to assess gender differences in terms of signaling behavior, such as facial expressions and language style; they will do so by studying how these differences are perceived by human interviewers in their indexing of personality and cognitive ability. For their research design, they rely on a large sample of men and women interviewees matched on standardized test scores of Graduate and Managerial Assessment (GMA), self-reported personality ratings, age, race, and ethnicity. The research will provide new opportunities for interdisciplinary training of students with an emphasis on recruiting underrepresented groups to work on this project. This research will provide information and guidance for developing bias-free machine-learning systems for personnel selection. By identifying and accounting for behavioral differences between genders that lead to predictive bias in machine learning selection systems, the proposed research will advance our understanding of the differences in gender expression of behaviors, methods for dealing with bias in machine learning, and bias reduction strategies in personnel selection and assessment.This project focuses on two scenarios of assessing interviewee attributes to train machine-learning algorithms: Algorithms trained on interviewee information (GMA test scores and self-reported personality), and algorithms trained on observer (interviewer) assessment of attributes. The matched sample ensures machine-learning model differences are not based on difference in underlying sample attributes. The two main goals of the project are: To understand gender differences in expressed behaviors and interviewer ratings (trained and untrained interviewers) using machine-learning techniques, and then to use that understanding to reduce predictive discrepancies between men and women by accounting for it in the models. The findings will have several significant societal impacts. They will improve our ability to predict and mitigate biases, bring to light new methodologies for mitigating bias in machine learning; and (provide strategies and tools for reducing social inequalities in employment outcomes. This research also has potential to advance both social science and machine learning. It will provide insights that can advance our understanding of social role theory by uncovering objective differences in behavior exhibited by men and women and how these behaviors are interpreted differently. Further, it can advance machine learning in developing new techniques for addressing bias at all stages of the machine-learning pipeline from instance selection and weighting, to model fitting, and then to model selection and optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目的是使用机器学习来理解和减轻访谈者评估中的偏见。研究人员将通过检查访谈中表达行为的性别差异来做到这一点；他们将专注于可能导致不同访谈者评估的行为。更具体地说，他们将使用无监督的视频访谈来评估信号行为的性别差异，例如面部表情和语言风格；他们将通过研究人格索引人格和认知能力的索引来研究这些差异。对于他们的研究设计，他们依靠大量的男女受访者样本，这些受访者与研究生和管理评估（GMA），自我报告的人格评级，年龄，种族和种族相匹配。这项研究将为学生跨学科培训提供新的机会，重点是招募代表性不足的小组从事该项目。这项研究将为开发用于人员选择的无偏见的机器学习系统提供信息和指导。通过识别和考虑到导致机器学习选择系统中预测性偏见的性别差异的行为差异，拟议的研究将提高我们对行为性别表达的差异的理解，处理机器学习中偏见的方法以及在人员选择中的偏见策略以及对人员选择的偏见策略和评估。这些项目的重点是培训机器属于培训者的情况：培训机器属于机器的情况： （GMA测试分数和自我报告的个性）和对观察者（访调员）属性评估培训的算法。匹配的样本确保机器学习模型差异不是基于基本样本属性的差异。该项目的两个主要目标是：使用机器学习技术了解表达行为的性别差异和表达行为和访谈者评级（经过培训和未经培训的访调员），然后利用该理解来减少男性和女性之间的预测性差异。这些发现将产生一些重大的社会影响。他们将提高我们预测和减轻偏见的能力，揭示减轻机器学习偏见的新方法；和（提供降低就业成果中社会不平等的策略和工具。这项研究还具有推进社会科学和机器学习的潜力。它将提供见解，通过揭示男性和女性所表现出的客观行为的客观差异来提高我们对社会角色理论的理解合适的，然后是建模选择和优化。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响审查标准来评估值得支持。

项目成果

Multimodal, Multiparty Modeling of Collaborative Problem Solving Performance

• DOI: 10.1145/3382507.3418877
• 发表时间: 2020-10
• 期刊: Proceedings of the 2020 International Conference on Multimodal Interaction
• 作者: Shree Krishna Subburaj;Angela E. B. Stewart;A. Rao;S. D’Mello

Toward Argument‐Based Fairness with an Application to AI‐Enhanced Educational Assessments

• DOI: 10.1111/jedm.12334
• 发表时间: 2022-06
• 期刊: Journal of Educational Measurement
• 影响因子: 1.3
• 作者: A. Huggins-Manley;Brandon M. Booth;S. D’Mello

Bias and Fairness in Multimodal Machine Learning: A Case Study of Automated Video Interviews

• DOI: 10.1145/3462244.3479897
• 发表时间: 2021-01-01
• 期刊: ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction
• 作者: Booth, Brandon M.;Hickman, Louis;D'mello, Sidney K.
• 通讯作者: D'mello, Sidney K.

A Conceptual Framework for Investigating and Mitigating Machine-Learning Measurement Bias (MLMB) in Psychological Assessment

用于调查和减轻心理评估中机器学习测量偏差 (MLMB) 的概念框架

• DOI: 10.1177/25152459211061337
• 发表时间: 2022
• 期刊: Advances in Methods and Practices in Psychological Science
• 影响因子: 13.6
• 作者: Tay, Louis;Woo, Sang Eun;Hickman, Louis;Booth, Brandon M.;D’Mello, Sidney
• 通讯作者: D’Mello, Sidney

Integrating Psychometrics and Computing Perspectives on Bias and Fairness in Affective Computing: A case study of automated video interviews

整合心理测量学和计算视角来看待情感计算中的偏见和公平：自动视频访谈的案例研究

• DOI: 10.1109/msp.2021.3106615
• 发表时间: 2021
• 期刊: IEEE Signal Processing Magazine
• 影响因子: 14.9
• 作者: Booth, Brandon M.;Hickman, Louis;Subburaj, Shree Krishna;Tay, Louis;Woo, Sang Eun;D'Mello, Sidney K.
• 通讯作者: D'Mello, Sidney K.