The Item Bank Calibration and Replenishment for Computerized Adaptive Testing in Small Scale Assessments: Method, Theory, and Application

小规模评估中计算机化自适应测试的题库校准和补充：方法、理论和应用

• 批准号: 2243044
• 负责人: Shiyu Wang
• 金额: $ 30万
• 依托单位: University of Georgia Research Foundation Inc
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2243044&HistoricalAwards=false
• 关键词: Item; Bank; Calibration; Replenishment; Computerized

This research project will advance statistical estimation methods for computerized adaptive testing (CAT) item bank calibration and replenishment in small-scale assessments. CAT has emerged as a powerful assessment tool and has been applied to the field of educational testing, quality of life measurement, health related measurement, and testing in industrial settings. Different from the traditional paper-pencil test, CAT allows for personalized assessment. However, the application of CAT remains limited in small-scale assessment scenarios, such as in classrooms or business daily routines. This project will develop a series of statistical estimation methods, theories, algorithms, and software aimed at accelerating the development of CAT and promoting its applications in personalized educational assessment and learning. The project will address a fundamental challenge in educational research: the lack of efficient statistical methods to cope with limited samples sizes and missing data, especially in low-stakes assessment settings. The outcomes are expected to advance the fields of education and psychology and to positively impact society. The investigators will apply the new methods to educational data sets and develop publicly available software that will be directly applicable for educators and applied researchers. Graduate and undergraduate students will participate in the conduct of this research. This research project will develop new statistical methods and theories to address the challenges of Item Response Theory (IRT) model calibration with small samples, sparse response data, and the lack of computationally efficient online estimation procedures with theoretical guarantee. The project will develop a dimension reduction method for a family of IRT models based on homogeneity structure learning, which belongs to a class of statistical and machine learning methods that aim to summarize useful information by discovering low-dimensional structures. An online inference approach that borrows the wisdom of some recent developments in stochastic gradient descent based online estimation and inference also will be developed. The investigators will validate these developed approaches through a set of simulation studies that mimic different aspects of small-scale assessments and analyze two real data sets from a computer-based classroom assessment and a large-scale CAT.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目将推进计算机化自适应测试（CAT）项目库校准和补充小规模评估的统计估计方法。 CAT已成为一种强大的评估工具，已应用于教育测试，生活质量测量，与健康相关的测量和工业环境中的测试领域。与传统的纸笔测试不同，CAT可以进行个性化评估。但是，在小规模评估方案中，例如在课堂或商务日常工作中，猫的应用仍有限。该项目将开发一系列统计估计方法，理论，算法和软件，以加速猫的发展并促进其在个性化的教育评估和学习中的应用。该项目将解决教育研究中的基本挑战：缺乏有效的统计方法来应对有限的样品尺寸和缺少数据，尤其是在低风险评估环境中。预计结果将推进教育和心理学领域，并积极影响社会。研究人员将将新方法应用于教育数据集并开发公开可用的软件，这些软件将直接适用于教育工作者和应用研究人员。研究生和本科生将参加这项研究的行为。该研究项目将开发新的统计方法和理论，以解决项目响应理论（IRT）模型校准的挑战，稀疏响应数据以及缺乏具有理论保证的计算高效的在线估计程序。该项目将基于同质性结构学习为IRT模型系列开发一种缩小方法，该方法属于一类统计和机器学习方法，旨在通过发现低维结构来总结有用的信息。 还将开发一种在线推理方法，该方法借用了基于随机梯度下降的在线估计和推理的最新发展的智慧。 研究人员将通过一系列模拟研究来验证这些开发的方法，这些研究模仿小规模评估的不同方面，并分析了基于计算机的课堂评估和大规模猫的两个真实数据集。这项奖项反映了NSF的法定任务，并具有。使用基金会的知识分子优点和更广泛的审查标准，被认为值得通过评估来支持。