Using Neural Networks for Automated Classification of Elementary Mathematics Instructional Activities

使用神经网络对基础数学教学活动进行自动分类

• 批准号: 2000487
• 负责人: Peter Youngs
• 金额: $ 150万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2000487&HistoricalAwards=false
• 关键词: Using; Neural; Networks; Automated; Classification

This research project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.In the last decade, there has been a tremendous increase in the use of video for preparing teachers and studying teaching quality. Prominent approaches to summative evaluation of teaching candidates and beginning teachers feature video recordings of instructional practices. In addition, large-scale research studies have included video of instruction. Teacher preparation programs increasingly use video in methods courses and for formative assessment purposes. Moreover, the growing use of interactive simulation in pre-service preparation and in-service professional development relies on having video recorded examples of exemplary instructional practices. Despite the significant growth in the use of video to measure and promote instructional quality in recent years, there remain some key challenges to employing it at scale. First, it is very time-consuming for trained human raters to view hundreds of hours of video recorded lessons. Second, financial costs and time demands increase with the use of multiple human raters per video. Third, manually cataloging, labeling, and indexing large volumes of classroom video for later viewing is time consuming. Recent advances in computer vision, machine learning, and deep learning may provide solutions to these challenges and could make the process of analyzing and scoring videos more efficient. In particular, deep learning has become the state-of-the-art choice in problems related to analyzing the content of video. This proposed NSF Core Research study will draw on videos of elementary mathematics instruction that were collected for two NSF-funded studies that featured the Mathematics-Scan (M-Scan) classroom observation tool. The research will use these videos to explore several ways that deep neural networks can be used to classify instructional activities in videos of math instruction.This study will advance knowledge and understanding by examining the degree to which three types of artificial neural networks can accurately classify (a) objects and (b) instructional activities in videos of elementary mathematics instruction. For example, it may be straightforward for neural networks to determine whether an elementary teacher is engaged in lecture vs. facilitating discussion with students. On the other hand, it may be harder for such networks to assess the ways teachers represent math content, the nature of their questions, and whether student gestures signify understanding. The research design concerns three aspects of using computer vision, machine learning, and deep learning: (a) the type of neural network, (b) the type of video label (i.e., object labels and instructional activity labels), and (c) the subject of math instruction (e.g., number and operations; patterns, functions, and algebra; geometry). A few types of neural networks have recently proven effective for video classification: convolutional neural networks (CNNs), long short-term memory (LSTM) neural networks, and hybrid CNN-LSTMs. Ultimately, this project is aimed at beginning to build systematic infrastructure for classifying videos of classroom instruction at scale in efficient and affordable ways. The findings will potentially have key implications for (a) large-scale research studies that feature videos of instruction and (b) pre-service teacher preparation programs, in-service professional development activities, and efforts to evaluate teaching candidates and practicing teachers. In particular, the results from this study will inform decisions about the types of neural networks that can be used to correctly classify videos of instruction and the practical limitations of using networks for this purpose.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目得到了 EHR 核心研究 (ECR) 计划的支持，该计划支持推进 STEM 学习和学习环境的基础研究、扩大 STEM 的参与以及 STEM 劳动力发展的工作。在过去的十年中，STEM 劳动力发展有了巨大的增长使用视频为教师做好准备并研究教学质量。对教学候选人和新教师进行总结性评估的主要方法是教学实践的视频记录。此外，大规模研究还包括教学视频。教师培训项目越来越多地在方法课程和形成性评估中使用视频。此外，在职前准备和在职专业发展中越来越多地使用交互式模拟，这依赖于录制示范性教学实践的视频示例。尽管近年来使用视频来衡量和提高教学质量的情况显着增长，但大规模使用视频仍然面临一些关键挑战。首先，对于经过培训的人类评分者来说，查看数百小时的视频录制课程非常耗时。其次，随着每个视频使用多个人工评分员，财务成本和时间需求会增加。第三，手动对大量课堂视频进行编目、标记和索引以供以后查看非常耗时。计算机视觉、机器学习和深度学习的最新进展可能会为这些挑战提供解决方案，并使视频分析和评分过程更加高效。特别是，深度学习已成为与分析视频内容相关的问题的最先进选择。这项拟议的 NSF 核心研究将利用基础数学教学视频，这​​些视频是为两项 NSF 资助的研究收集的，这些研究以数学扫描 (M-Scan) 课堂观察工具为特色。该研究将利用这些视频探索深度神经网络可用于对数学教学视频中的教学活动进行分类的几种方法。这项研究将通过检查三种类型的人工神经网络准确分类的程度来增进知识和理解（初等数学教学视频中的 a) 物体和 (b) 教学活动。例如，神经网络可能很容易确定小学教师是否正在讲课或是否促进与学生的讨论。另一方面，此类网络可能更难评估教师表达数学内容的方式、问题的性质以及学生的手势是否表示理解。研究设计涉及使用计算机视觉、机器学习和深度学习的三个方面：(a) 神经网络的类型，(b) 视频标签的类型（即对象标签和教学活动标签），以及 (c)数学教学的主题（例如，数字和运算；模式、函数和代数；几何）。最近，几种类型的神经网络已被证明对视频分类有效：卷积神经网络 (CNN)、长短期记忆 (LSTM) 神经网络和混合 CNN-LSTM。最终，该项目的目标是开始构建系统基础设施，以高效且负担得起的方式对大规模课堂教学视频进行分类。研究结果可能对以下方面产生重大影响：（a）以教学视频为特色的大规模研究；（b）职前教师准备计划、在职专业发展活动以及评估教学候选人和执业教师的努力。特别是，这项研究的结果将为有关可用于正确分类教学视频的神经网络类型以及为此目的使用网络的实际限制的决策提供信息。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。

项目成果

TAA-GCN: A temporally aware adaptive graph convolutional network for age estimation.

TAA-GCN：用于年龄估计的时间感知自适应图卷积网络。

• 发表时间: 2023-02
• 期刊: Pattern recognition
• 影响因子: 8
• 作者: Korban, M.;Youngs, P.;Acton, S. T.
• 通讯作者: Acton, S. T.

A multi-modal transformer network for action detection.

用于动作检测的多模态变压器网络。

• 发表时间: 2023-10
• 期刊: Pattern recognition
• 影响因子: 8
• 作者: Korban, M.;Youngs, P.;Acton, S
• 通讯作者: Acton, S