FairFL-MC: A Metacognitive Calibration Intervention Powered by Fair and Private Machine Learning

FairFL-MC：由公平和私人机器学习支持的元认知校准干预

• 批准号: 2202481
• 负责人: Dong Wang
• 金额: $ 85万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202481&HistoricalAwards=false
• 关键词: FairFL; MC; Metacognitive; Calibration; Intervention

Students often have difficulty estimating their own level of knowledge. The goal of this project is to research ways to improve students' ability to estimate their knowledge, using a student support system consisting of short training exercises that will be personalized with artificial intelligence (AI) methods. While there is abundant research on AI methods in educational contexts, such projects rarely consider some of the key social and human factors, such as privacy and fairness, that are needed for widespread adoption of personalized educational software. This project addresses these issues with a novel decentralized AI framework that is specifically for education contexts. The project framework will enable researchers to create AI systems that provide feedback to students as part of their training exercises, all without directly accessing their data and while also training the AI system to reduce biases related to key aspects of students' identity, such as their demographics. The training exercises will include educational activities where students estimate their test scores, receive feedback from the AI system, and reflect on their knowledge. The privacy and fairness capabilities of the project framework will transform postsecondary online learning, which is poised to benefit from emerging AI-driven learning technologies but has yet to fully realize these benefits. The project will directly benefit students participating in the research as they will improve their knowledge estimation skills, prepare more effectively for tests in class, and learn about potential privacy violations and AI biases. Given the fairness focus of the project, the team of researchers will pay special attention to benefits for students from groups traditionally underrepresented in STEM (Science, Technology, Engineering, and Mathematics), ensuring that the AI-powered framework is equally helpful for them and that their perspectives on privacy and fairness receive special attention.This project will advance AI research by incorporating, both, a strict privacy guarantee for student data and fairness considerations across multiple student demographic groups. Additionally, it will advance education research by determining how effective preemptive feedback is for improving knowledge estimation skills, and will examine the mechanism by which preemptively improving knowledge estimation influences academic outcomes. In particular, the project will achieve four research objectives through interdisciplinary innovations in both learning sciences and technology. First, the team will determine how much students' metacognitive calibration can be improved via AI-powered preemptive feedback, which may be perceived differently by students than post hoc feedback. Second, the project will expand theoretical understanding of metacognitive calibration and calibration interventions by studying the mechanism by which the intervention in the project works. Third, the team will address the fundamental tradeoff between the fairness and accuracy of AI models via an innovative federated learning model. Fourth, the team will evaluate the AI framework on real-world education datasets and compare its performance with the state-of-the-art baselines in terms of protecting privacy and mitigating bias. The project team will disseminate results of the project through workshops, publications, and interactive activities, and will train undergraduate and graduate students from diverse backgrounds throughout the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

学生通常很难估计自己的知识水平。该项目的目的是使用由简短的培训练习组成的学生支持系统来提高学生估计知识的能力，这些练习将通过人工智能（AI）方法进行个性化。尽管在教育环境下对AI方法进行了丰富的研究，但此类项目很少考虑一些主要采用个性化教育软件所需的关键社会和人为因素，例如隐私和公平。该项目通过专门针对教育环境的新型分散的AI框架来解决这些问题。该项目框架将使研究人员能够创建AI系统，以作为他们的培训练习的一部分提供反馈，而无需直接访问他们的数据，同时还训练AI系统以减少与学生身份的关键方面相关的偏见，例如人口统计。培训练习将包括教育活动，学生估计考试成绩，从AI系统获得反馈并反思他们的知识。项目框架的隐私和公平能力将改变学院的在线学习，该学习有望从新兴的AI驱动学习技术中受益，但尚未完全实现这些好处。该项目将直接受益于参加研究的学生，因为他们将提高知识估算技能，更有效地为课堂测试做好准备，并了解潜在的侵犯隐私行为和AI偏见。鉴于该项目的公平重点，研究人员团队将特别关注STEM（科学，技术，工程和数学）传统上代表性不足的团队的学生的利益，以确保AI驱动的框架对他们和他们同样有用他们对隐私和公平性的看法受到了特别的关注。该项目将通过对多个学生人口组的学生数据和公平考虑因素进行严格的隐私保证来提高AI研究。此外，它将通过确定提高知识估计技能的有效先发制反馈的有效性，并将研究抢先提高知识估算影响学术成果的机制，从而提高教育研究。特别是，该项目将通过学习科学和技术的跨学科创新实现四个研究目标。首先，团队将通过AI驱动的先发制人反馈来确定可以改善学生的元认知校准数量，学生对学生的看法与事后的反馈不同。其次，该项目将通过研究项目的干预措施的机制来扩展对元认知校准和校准干预措施的理论理解。第三，该团队将通过创新的联合学习模型解决AI模型的公平和准确性之间的基本权衡。第四，该团队将评估现实世界教育数据集的AI框架，并将其性能与最新的基线相提并论，以保护隐私和减轻偏见。项目团队将通过研讨会，出版物和互动活动来传播项目的结果，并将在整个项目中培训来自不同背景的本科生和研究生。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的评估来支持的。智力优点和更广泛的影响审查标准。