EAGER: Volition Based Anticipatory Control for Time-Critical Brain-Prosthetic Interaction

EAGER：基于意志的预期控制，用于时间关键的大脑-假体交互

• 批准号: 1550397
• 负责人: Gil Weinberg
• 金额: $ 17.88万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-08-15 至 2017-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1550397&HistoricalAwards=false
• 关键词: EAGER; Volition; Based; Anticipatory; Control

This exploratory project focuses on developing algorithms that will allow the PI's previously implemented prototype drumming prosthesis, which was developed in an effort to help an injured teen, to anticipate human physical actions based on an analysis of EEG signals so that it can respond mechanically in a timely manner. The goal is to enable the enhanced prosthesis to detect volition, the cognitive process by which an individual decides on and commits to a particular course of action hundreds of milliseconds before the action actually takes place, in order to foresee the drummer's actions and achieve sub-second synchronization between artificial and biological limbs, thereby leading to improved performance in a time-sensitive domain where asynchronous operations of more than a few milliseconds are noticeable by listeners. Project outcomes will include cognitive models and technical approaches that will be of great value for improving efficiency and fluency in a wide range of human-robot and human-prosthesis interaction scenarios, from construction tasks where humans and robots collaborate to achieve common goals, to time-critical tasks such as in hospital operating rooms or space stations where humans operate artificial robotic limbs. The work will also lead to creation of a volition trials database that will be documented and shared with the broad community of brain scholars and brain-machine interface researches. And the project will have additional broad impact by supporting students in the Robotic Musicianship group at Georgia Tech as it transitions from its previous focus on robotic musicianship into the fields of prosthetic and human augmentation.Prior studies of volition have shown that across multiple repetitions of (real or imagined) motor activity one can derive the Event-Related-Potential (ERP) associated with the intent to move the hand, up to a few seconds prior to the generation of the movement. Additionally, studies of mirror neurons have shown that observing a motor activity can trigger sets of cells in the brain that replicate the activity depicted when a subject is engaged in the action itself. In this project the PI will build on such findings to develop new pattern recognition algorithms for EEG signal analysis in an effort to identify volition and design new anticipatory algorithms for brain-machine interfaces that reduce latency and allow for synchronization at the millisecond level. The work will be carried out in stages. The PI will first collect EEG data from a large number of experimental trials where participants are engaged in a voluntary motor action. The data will be studied to detect patterns indicative of volition activity from electrodes monitoring both the motor and pre-motor cortices (SMA and pre-SMA), and also to isolate the neural correlates of imagined vs. real movement. A variety of general purpose machine learning classifiers, as well as music focused feature extraction techniques, will be used to distinguish between anticipatory patterns of activity preluding an action (volition) and patterns generated when the action is indeed manifested. As part of the analysis the PI will attempt to acquire an understanding of the delta times between volition and action under different conditions, and he will develop a repeatability / reliability matrix to be utilized for synchronization in the next stage of the work, in which the PI will develop a "latency compensation engine" that generates robotic drum hits at the exact anticipated action time, compensating for mechanical latencies while taking into account the projected delta time between volition and action. Multi-modal integration with data from other sensors (EMG, microphones, proximity, etc.) will be exploited to correct errors is detection and classification. Finally, success of the new algorithms will be evaluated using both objective and subjective measures by having the amputee drummer perform a series of musical tasks with the robotic arm.

这个探索性项目的重点是开发算法，使 PI 之前实现的击鼓假肢原型（该假肢是为了帮助受伤的青少年而开发的）能够根据脑电图信号分析来预测人类的身体动作，从而能够在及时的方式。 目标是使增强型假肢能够检测意志，即个人在行动实际发生之前数百毫秒决定并致力于特定行动方针的认知过程，以便预见鼓手的行动并实现子目标。假肢和生物肢体之间的第二次同步，从而提高了时间敏感领域的性能，在该领域中，听众可以注意到超过几毫秒的异步操作。 项目成果将包括认知模型和技术方法，对于提高各种人机和人机交互场景的效率和流畅性具有重要价值，从人与机器人协作实现共同目标的施工任务，到时间- 关键任务，例如在医院手术室或空间站中人类操作人造机器人肢体。 这项工作还将创建一个意志试验数据库，该数据库将被记录下来并与广大脑学者和脑机接口研究人员共享。 该项目还将通过支持佐治亚理工学院机器人音乐小组的学生来产生更广泛的影响，因为该小组从之前的重点是机器人音乐转向假肢和人体增强领域。先前对意志的研究表明，通过多次重复（真实的或想象的）运动活动，人们可以在运动产生之前几秒钟导出与移动手的意图相关的事件相关电位（ERP）。 此外，镜像神经元的研究表明，观察运动活动可以触发大脑中的一组细胞，这些细胞会复制受试者参与动作本身时所描绘的活动。 在这个项目中，PI 将基于这些发现开发用于脑电图信号分析的新模式识别算法，以努力识别意志并为脑机接口设计新的预期算法，以减少延迟并允许毫秒级的同步。 这项工作将分阶段进行。 PI 将首先从大量实验中收集脑电图数据，其中参与者参与自愿运动动作。 将研究这些数据，以检测来自监测运动皮质和前运动皮质（SMA 和 pre-SMA）的电极的意志活动模式，并分离想象与真实运动的神经相关性。 各种通用机器学习分类器以及以音乐为中心的特征提取技术将用于区分动作（意志）前的预期活动模式和动作确实表现出来时生成的模式。 作为分析的一部分，PI 将尝试了解不同条件下意志和行动之间的增量时间，他将开发一个可重复性/可靠性矩阵，用于下一阶段工作的同步，其中PI 将开发一种“延迟补偿引擎”，在精确的预期动作时间生成机器人鼓击，补偿机械延迟，同时考虑到意志和动作之间的预计增量时间。 将利用与其他传感器（肌电图、麦克风、接近度等）数据的多模式集成来纠正检测和分类中的错误。 最后，将通过让截肢鼓手用机械臂执行一系列音乐任务，使用客观和主观措施来评估新算法的成功。