HCC-Small: RSVP IconCHAT - A Brain Computer Interface for Icon-based Communication

HCC-Small：RSVP IconCHAT - 用于基于图标的通信的脑机接口

• 批准号: 0914808
• 负责人: Deniz Erdogmus
• 金额: $ 50万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0914808&HistoricalAwards=false
• 关键词: HCC; Small; RSVP; IconCHAT; Brain

In this project the PI will address the challenge of empowering people with severe motor and speech impairments (SMSI) to socialize through written and spoken language, by increasing communication rate through a novel and intuitive computer interface. Available augmented communication technologies for the SMSI population typically yield speeds on the order of just one word per minute (based on clinical experience). The PI's objective is to develop an EEG-based brain interface technology based on an intuitive icon-based language generation framework, RSVP iconCHAT, which will achieve increased communication rates for the target population. This technology will exhibit three essential features: rapid serial visual presentation (RSVP) of icons that represent words; a large-vocabulary natural language model with the capability for accurate predictions of intended text in order to control the upcoming sequence of icons to be shown to the subject for confirmation in the RSVP paradigm; and an intent detection mechanism that fuses information from multichannel electroencephalography (EEG) and the generative probabilistic language model. Advanced statistical signal processing, machine learning, and natural language modeling techniques will be employed to achieve communication rates over an order of magnitude higher than the current state-of-the-art. The project will also contribute novel techniques and algorithms for synchronous brain interface design, particularly single-trial ERP detection. Both the brain interface and language model components will learn from previous interactions with the user and exhibit robust cooperative learning behavior in order to maximize language throughput. A Bayesian and information theoretic foundation will support adaptability. The PI notes that his approach is innovative along three dimensions: an intuitive icon-based language representation combined with context-dependent language models will be employed for message construction; a noninvasive brain computer interface that is user-adaptive will be developed and employed to interface with the icon-based platform; and methods for probabilistic information fusion between the brain activity measured by the BCI and the predictive language model will be developed.Broader Impacts: There exists a significant SMSI population due to various reasons such as cerebral palsy (CP), neuromuscular disease (Amyotrophic Lateral Sclerosis, ALS), and severe spinal cord injury leading to locked-in syndrome (LIS). These communities rely on inefficient modes of communication that limit the user's ability to generate acceptable communication rates. Successful achievement of this project's goals will not only provide the target population with an improved face-to-face communication experience with their able-bodied communication partners, but will also enable control of their environment and access to information. In addition, the work will contribute to information fusion from different modalities, optimal data dimensionality reduction, single-trial ERP detection, and human computer communication through a novel interface. Data collected in experiments will be made available to other researchers in order to accelerate verification of outcomes and dissemination of results.

在该项目中，PI将通过通过新颖而直觉的计算机接口提高沟通速度来解决通过书面和口语进行社交的挑战（SMSI）通过书面和口语进行社交的挑战。 SMSI人群的可用增强通信技术通常以每分钟一个单词的顺序（基于临床经验）产生速度。 PI的目标是基于基于直观的图标的语言生成框架RSVP Iconchat开发基于EEG的大脑界面技术，该框架将提高目标人群的通信率。 该技术将展示三个基本特征：代表单词的图标快速串行视觉呈现（RSVP）；具有准确预测预期文本的能力的大型Vocabulary自然语言模型，以控制将显示给主题的即将到来的图标序列，以在RSVP范式中进行确认；以及一种意图检测机制，该机制融合了多通道脑电图（EEG）和生成概率语言模型的信息。 先进的统计信号处理，机器学习和自然语言建模技术将采用比当前最新技术高的数量级实现通信率。 该项目还将为同步脑界面设计，尤其是单审ERP检测提供新的技术和算法。 大脑界面和语言模型组件都将从以前与用户的互动中学习，并表现出强大的合作学习行为，以最大程度地提高语言吞吐量。 贝叶斯和信息理论基础将支持适应性。 PI指出，他的方法是沿三个维度创新的：将使用基于直观的图标语言表示与上下文依赖语言模型进行消息构建；将开发并使用具有用户自适应的非侵入性大脑计算机接口与基于图标的平台进行交互；将开发由BCI测量的大脑活动与预测语言模型之间的概率信息融合的方法。Broader的影响：由于各种原因，诸如大脑麻痹（CP），神经肌肉肿瘤疾病（神经营养性的侧面硬皮病，Als）和严重的脊柱或然脊柱损伤的原因，存在大量的SMSI人群。 这些社区依靠效率低下的通信模式，从而限制了用户产生可接受的通信率的能力。 成功实现该项目的目标不仅将为目标人群提供与能力健全的沟通伙伴的面对面交流经验的改善，而且还将能够控制其环境和获取信息。 此外，这项工作将有助于来自不同方式，最佳数据维度降低，单次试验ERP检测和人类计算机通信的信息融合。 在实验中收集的数据将提供给其他研究人员，以加速结果的验证和结果传播。