Research: Faculty Assessment Mental Models in Engineering Education

研究：工程教育中的教师评估心理模型

基本信息

批准号：
2113631
负责人：
Andrew Katz
金额：
$ 34.92万
依托单位：
Virginia Polytechnic Institute and State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-09-01 至 2025-02-28
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113631&HistoricalAwards=false
关键词：
Research Faculty Assessment Mental Models

项目摘要

The process of forming engineers is an iterative one that requires feedback to indicate developmental progress and identify areas for improvement. A primary source of feedback comes from assessment, which can play many roles in engineering education: a signal to students of what they do and do not understand about a concept; feedback to instructors about students’ conceptual understanding as well as what may or may not be working regarding their own teaching approaches; and information to administrators and prospective employers evaluating students’ abilities. Although assessments function as a linchpin in the formation of engineers, it is unclear how faculty members - i.e., the individuals typically designing and implementing these assessments - think about this pertinent signaling mechanism. Because faculty members often have autonomy in making course decisions, understanding how they think about assessments is essential to establish the foundation on future efforts in promoting diverse and improved assessment approaches in engineering education. To better understand how faculty think about and make decisions on assessment, we have designed a three-phase study that uses interviews, surveys, and natural language processing techniques to gather extensive data from a diverse sample of faculty who will undoubtedly have diverse views on students and assessment. The outcomes of this study will include characterizing faculty mental models of assessment and how those models inform instructional decisions. In developing these outcomes, we will also identify potential biases, misconceptions, and problematic, systemic patterns in assessment implementation. The knowledge generated through this project will inform better faculty training and policies to advance this vital area in the formation of engineers.To characterize faculty mental models and how they inform decision-making regarding assessment, we will engage in a three-phased, multi-method study. Drawing on exploratory interviews, Phase 1 will map the landscape of mental models that faculty members might have related to assessment. Phase 2 will draw on the initial interviews and add experience sampling methods to expand this mapping to connect those mental models with decisions made by faculty related to assessment over the course of a semester. Phase 3 will use a survey to expand the study sample in order to support inferential statements about the population of engineering faculty members in US engineering education ecosystems more broadly with regard to mental models and decisions related to assessment. This study will make several important contributions regarding intellectual merit. First, our study will identify a range of mental models that faculty engage in assessment-related decision-making, providing a view of the current state in engineering education. Second, building on these findings, we will illuminate the connections between these central instructional decisions and deeper perspectives that faculty members wield. Third, from a methodological perspective, we will implement ground-breaking methods that combine experience sampling methods (ESM) and natural language processing (NLP) and provide a model for other researchers in engineering education to do the same.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

培养工程师的过程是一个迭代的过程，需要反馈来表明发展进度并确定需要改进的领域。反馈的主要来源来自评估，评估可以在工程教育中发挥多种作用：向学生发出他们所做的事情和正在做的事情的信号。不理解某个概念；向教师反馈学生的概念理解以及他们自己的教学方法可能有效或无效的信息；以及向管理人员和未来雇主评估学生能力的信息。工程师的组建，目前尚不清楚教职人员如何 -即通常设计和实施这些评估的个人 - 考虑这种相关的信号机制，因为教师在做出课程决策时通常具有自主权，因此了解他们如何看待评估对于为未来促进多样化和改进的评估奠定基础至关重要。为了更好地了解教师如何思考和做出评估决策，我们设计了一项三阶段研究，使用访谈、调查和自然语言处理技术从不同的教师样本中收集大量数据，这些数据无疑会对学生和评估结果有不同的看法。将包括描述教师评估的心理模型以及这些模型如何影响教学决策。在制定这些结果时，我们还将识别评估实施中潜在的偏见、误解和有问题的系统模式。通过该项目产生的知识将为更好的教师培训提供信息。为了描述教师的心理模型以及它们如何为评估决策提供信息，我们将利用探索性访谈进行三阶段、多方法的研究，第一阶段将进行。绘制心理模型的图景第二阶段将利用最初的访谈并添加经验抽样方法来扩展此映射，以将这些心理模型与第三阶段将使用的与评估相关的教师做出的决策联系起来。一项扩大研究样本的调查，以更广泛地支持关于美国工程教育生态系统中工程教员群体在心理模型和与评估相关的决策方面的推论。，我们的研究将确定教师参与的一系列心理模型其次，基于这些发现，我们将阐明这些核心教学决策与教师所持的更深入观点之间的联系。第三，从方法论的角度来看。，我们将实施结合经验采样方法（ESM）和自然语言处理（NLP）的突破性方法，并为工程教育领域的其他研究人员提供一个模型。该奖项反映了 NSF 的法定使命，并被认为是值得的通过利用基金会的智力优势进行评估来提供支持，更广泛的影响审查标准。