Making Math Tutors More Engaging and Effective through Interaction Design Patterns and Educational Data Mining

通过交互设计模式和教育数据挖掘使数学导师更具吸引力和效率

• 批准号: 1252297
• 负责人: Ryan Baker
• 金额: $ 148.09万
• 依托单位: Teachers College, Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1252297&HistoricalAwards=false
• 关键词: Making; Math; Tutors; More; Engaging

This project develops and validates interaction design patterns, structured definitions of high-quality design solutions that can be applied at scale, that can be used to design more effective and engaging online mathematics problems. This research is conducted in the context of ASSISTments, free online mathematics software used by middle school students nationwide. The patterns are designed using data from 20,000 mathematics problems previously used by thousands of students (generating millions of data points), and validated through a set of forty small-scale randomized controlled trials conducted via automated experimentation, conducted online in American classrooms nationwide. The project is a partnership among Teachers College, Columbia University, Carnegie Mellon University, and Worcester Polytechnic Institute. The project makes two types of contributions. The first is are basic discoveries as to which types of content lead to the best learning and engagement in online problem-solving. The second is a set of guidelines for the design of online learning that can be applied by teachers and other content creators to produce educationally effective and engaging online mathematics problems.The project is accomplishing these goals using the following procedure. First, they are studying the design features present in existing ASSISTments mathematics problems, by hand-labeling design features on a small sub-set of problems and then using educational data mining to replicate the hand-labels at scale. These design features include features relevant to interface design, domain content, and pedagogical strategies. They then apply previously developed and validated automated detectors of student learning, engagement, and affect to log files of students solving mathematics problems in ASSISTments. They use association rule mining to determine which combinations of design features lead to better learning, engagement, and affect, and build on these findings to develop interaction design patterns that communicate effective solutions which combine these data features. The design patterns are validated through a set of forty small-scale randomized controlled trials conducted via automated experimentation, where forty mathematics problems are improved using the design patterns. Each improved problem is studied in a random sample of 200 students (drawn from the full population of students currently using ASSISTments as part of their regular curriculum), who receive the problem as part of their regular classroom activities. They statistically assess the impact of these modified problems on learning and engagement using the outputs of automated detectors as dependent measures. This project results in increasing the effectiveness and engagingness of the mathematics problems in the ASSISTments system, and identifies design patterns that can be applied to improve all of the content in ASSISTments. 50,000 students a year use the ASSISTments system, including large numbers of students from traditionally under-represented populations. More broadly, the proposed project is producin a generalizable and precise approach for the creation of more effective and engaging online learning. The design patterns developed are likely to be useful for improving the design of the range of online problem-solving systems used increasingly in American mathematics education.

该项目开发并验证了交互设计模式，可以大规模应用的高质量设计解决方案的结构化定义，这些定义可用于设计更有效和引人入胜的在线数学问题。这项研究是在全国初中学生使用的辅助工具，免费的在线数学软件的背景下进行的。这些模式是使用来自数千名学生（生成数百万个数据点）先前使用的20,000个数学问题的数据设计的，并通过在全国范围内在美国教室在线进行的四十组小规模的随机对照试验进行了验证。该项目是教师学院，哥伦比亚大学，卡内基·梅隆大学和伍斯特理工学院的合作伙伴关系。该项目做出了两种类型的贡献。首先是关于哪种类型的内容导致在线解决问题的最佳学习和参与度的基本发现。第二个是一套在线学习设计的指南，教师和其他内容创建者可以应用，以产生有效且引人入胜的在线数学问题。该项目正在使用以下程序实现这些目标。首先，他们正在研究现有辅助数学问题中存在的设计功能，通过一小部分问题的手工标记设计功能，然后使用教育数据挖掘来大规模复制手工标签。这些设计功能包括与界面设计，域内容和教学策略相关的功能。然后，他们应用了先前开发并验证的学生学习，参与度和影响力的自动检测器，以在辅助中解决数学问题的学生文件的日志文件。他们使用关联规则挖掘来确定设计功能的哪些组合可以提高更好的学习，参与和影响，并以这些发现为基础，以开发互动设计模式，以传达结合这些数据功能的有效解决方案。设计模式通过通过自动实验进行的40组随机对照试验进行了验证，其中使用设计模式改善了40个数学问题。在200名学生的随机样本中研究了每个改进的问题（从当前使用助理作为常规课程的一部分的学生人群中汲取），他们将问题作为常规课堂活动的一部分。他们从统计上评估了这些修改后的问题对学习和参与的影响，使用自动检测器的输出作为依赖措施。该项目导致辅助系统中数学问题的有效性和引人入胜，并确定可用于改善辅助中所有内容的设计模式。每年有50,000名学生使用辅助系统，其中包括来自传统代表性不足的人群的大量学生。更广泛地说，拟议的项目是一种可创建更有效和引人入胜的在线学习的可普遍和精确的方法。开发的设计模式可能有助于改善美国数学教育中越来越多地使用的在线问题解决系统的设计。