PFI: AIR-TT: Commercializing a new genre of Intelligent Science Stations for informal and formal learning

PFI：AIR-TT：将新型智能科学站商业化，用于非正式和正式学习

• 批准号: 1701107
• 负责人: Ken Koedinger
• 金额: $ 20万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-07-15 至 2022-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1701107&HistoricalAwards=false
• 关键词: PFI; AIR; TT; Commercializing; new

This PFI: AIR Technology Translation project aims to build and commercialize a reusable mixed-reality platform that engages children in scientific experimentation and uses computer vision and automated feedback to help them effectively learn scientific principles. The project focuses on translating prior science and technology research that produced the first Intelligent Science Station, called EarthShake. EarthShake is a mixed-reality educational game in which children discover physical properties of balance by running experiments with block towers on a simulated earthquake table. This project has the potential to have substantial societal, commercial and educational impacts. It aims to impact millions of children and families that spend time in Children's and Science Museums, libraries, schools, and indoor play spaces. It will transform traditional hands-on exhibits and play spaces into intelligent learning environments that foster children's curiosity and improve their learning in an engaging and collaborative way. It will also help to transform the traditional educational system in schools, introducing an interactive, engaging, mixed-reality system that can help students understand the underlying evidence for scientific principles, not just memorize them. With over 270 Children's and Science Museums in the US alone, over 800 malls with play spaces for children, 100s of indoor play/learning spaces, over 4,000 clubs and after-school programs, 119,487 libraries, 36,767 private and charter elementary schools and 103,460 elementary schools, this project can have a commercial impact in addition to societal and educational benefit.The system uses computer vision to follow children's progress and provide intelligent guidance in a predict-observe-explain scientific inquiry process. Experiments demonstrate that grades K to 3 children learn much more from EarthShake than from an otherwise identical flatscreen tablet or laptop implementation showing as much as a 5 times greater pre to post increase on tests of prediction, explanation, and stable tower building. The initial prototype will be translated into a reusable platform for Intelligent Science Stations and demonstrated in scalable development of multiple instances of three games, EarthShake, RaceCars, and BalloonScale. The technical challenges are to generalize the vision algorithm to work across the variety of objects and configurations in these games and create modular hardware components that provide for scale in development, distribution, and new products. This project draws on technical expertise in computer vision/AI, mechatronics, tangible interfaces, physical computing, human/child-computer interaction, and educational technology.Intelligent Science Stations bridge the advantages of physical and virtual worlds to improve children's science and inquiry learning in an enjoyable and collaborative way and can be commercialized to do so in various settings, indoor play spaces, museums, libraries, and schools. The project contributes to the general robotics challenge by creating an automated intelligent tutoring system operating in the physical 3D world, incorporating sensors and perceptual algorithms to track what learners are doing, actuators to manipulate the scientific apparatus, and an intelligent dialogue system to orchestrate interaction. It addresses technical challenges of creating a generalizable vision algorithm and a modular physical hardware platform that allows customers to easily switch between different applications. It will create new knowledge about how to develop a general and reusable mixed-reality platform that integrates scientific apparatus including mechatronic components (e.g,. sensors, actuators, servo motors, physical experimental set-up) with an intelligent tracking system (including hardware and software components with an AI vision algorithm).

PFI：AIR 技术翻译项目旨在构建一个可重复使用的混合现实平台并将其商业化，让儿童参与科学实验，并使用计算机视觉和自动反馈来帮助他们有效地学习科学原理。该项目的重点是转化先前的科学和技术研究成果，这些研究成果产生了第一个名为“EarthShake”的智能科学站。 EarthShake 是一款混合现实教育游戏，孩子们可以通过在模拟地震台上用积木塔进行实验来发现平衡的物理特性。该项目有可能产生重大的社会、商业和教育影响。它旨在影响数百万在儿童和科学博物馆、图书馆、学校和室内游乐空间度过时光的儿童和家庭。它将把传统的实践展览和游戏空间转变为智能学习环境，培养孩子们的好奇心，并以互动和协作的方式提高他们的学习能力。它还将有助于改变学校的传统教育系统，引入交互式、引人入胜的混合现实系统，可以帮助学生理解科学原理的潜在证据，而不仅仅是记住它们。 仅在美国就有 270 多个儿童和科学博物馆、800 多个设有儿童游乐空间的购物中心、100 个室内游乐/学习空间、4,000 多个俱乐部和课外活动、119,487 个图书馆、36,767 所私立和特许小学以及 103,460 所小学对于学校来说，该项目除了社会和教育效益之外，还可以产生商业影响。该系统使用计算机视觉来跟踪儿童的进步，并在预测-观察-解释的科学探究过程中提供智能指导。实验表明，K 到 3 年级的孩子从 EarthShake 中学到的东西比从其他相同的平板平板电脑或笔记本电脑中学到的东西要多得多，在预测、解释和稳定塔楼建造测试中，前后的增幅高达 5 倍之多。 最初的原型将被转化为智能科学站的可重复使用平台，并在 EarthShake、RaceCars 和 BalloonScale 三个游戏的多个实例的可扩展开发中进行演示。技术挑战在于将视觉算法推广到这些游戏中的各种对象和配置上，并创建模块化硬件组件，以实现开发、发行和新产品的规模化。该项目利用了计算机视觉/人工智能、机电一体化、有形界面、物理计算、人机交互和教育技术等方面的技术专业知识。智能科学站弥合了物理世界和虚拟世界的优势，以提高儿童的科学和探究学习能力。这是一种令人愉快和协作的方式，并且可以在各种环境、室内游乐空间、博物馆、图书馆和学校中进行商业化。该项目通过创建在物理 3D 世界中运行的自动化智能辅导系统，结合传感器和感知算法来跟踪学习者正在做什么，使用执行器来操纵科学仪器，以及智能对话系统来协调交互，从而应对一般机器人挑战。 它解决了创建通用视觉算法和模块化物理硬件平台的技术挑战，使客户能够在不同的应用程序之间轻松切换。它将创造关于如何开发通用且可重复使用的混合现实平台的新知识，该平台将包括机电一体化组件（例如传感器、执行器、伺服电机、物理实验装置）在内的科学仪器与智能跟踪系统（包括硬件和具有人工智能视觉算法的软件组件）。

项目成果

Active Learning is About More Than Hands-On: A Mixed-Reality AI System to Support STEM Education

主动学习不仅仅是动手：支持 STEM 教育的混合现实人工智能系统

• DOI: 10.1007/s40593-020-00194-3
• 发表时间: 2020
• 期刊: International Journal of Artificial Intelligence in Education
• 影响因子: 4.9
• 作者: Yannier, Nesra;Hudson, Scott E.;Koedinger, Kenneth R.
• 通讯作者: Koedinger, Kenneth R.

Intelligent science exhibits: Transforming hands-on exhibits into mixed-reality learning experiences

智能科学展览：将动手展览转变为混合现实学习体验

• DOI: 10.1080/10508406.2022.2032071
• 发表时间: 2022
• 期刊: Journal of the Learning Sciences
• 影响因子: 3.8
• 作者: Yannier, Nesra;Crowley, Kevin;Do, Youngwook;Hudson, Scott E.;Koedinger, Kenneth R.
• 通讯作者: Koedinger, Kenneth R.