CHS: Small: Collaborative Research: A Graph-Based Data Fusion Framework Towards Guiding A Hybrid Brain-Computer Interface

CHS：小型：协作研究：基于图的数据融合框架指导混合脑机接口

• 批准号: 2006012
• 负责人: Yalda Shahriari
• 金额: $ 30.98万
• 依托单位: University of Rhode Island
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2006012&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Graph

Major advances in non-invasive brain-computer interfaces (BCIs) have enriched the lives of persons with certain disabilities by providing them with alternative means of communication. However, current systems rely heavily on unimodal techniques that limit both their performance and our understanding of the integrated neural dynamics essential to properly explain multiscale neural functions. To address this issue it has been proposed to employ hybrid (multimodal) BCIs, but attempts to date to utilize the complementary benefits of multiple modalities through simple combinations (e.g., concatenation of feature sets from two neuroimaging modalities) have yielded only incremental advances; generalizable computational data-driven approaches for the fusion of multimodal signals to efficiently and simultaneously extract complementary information from multiple signals of interest remain lacking. This research will explore an innovative approach to a hybrid non-invasive BCI system that capitalizes on the complementary physiological features that can be obtained from electrical and hemodynamic neural signals using EEG and fNIRS respectively, with the help of a graph-based data fusion framework. Project outcomes will include novel signal processing pipelines and lay the foundation for practical BCI techniques for mainstream user applications. In addition to the project's potential societal impacts, the team will focus on broadening participation in STEM and will also engage students from K-12 through the graduate level.The research will involve three main thrusts. A novel graph theoretical multimodal data fusion framework will be developed to systematically capture complex topological features of hybrid patterns and user intentions during a dual-task interaction that concurrently modulates electrical and hemodynamic responses of interest. Because multimodal techniques create inherently complementary attributes in terms of both spatiotemporal resolution and information content, the framework will aim to capture the corresponding complementary synergistic topological features from the complex hybrid patterns hidden in EEG and fNIRS signals for the high-level abstraction of user intentions. The framework will be evaluated on non-communicative individuals by optimizing parameters and channels containing the highest mutual information, in real-world settings. Finally, a conceptually new hybrid subspace-based filter will be proposed to maximize the distance between two classes of hybrid data and enhance classification performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

非侵入性脑部计算机界面（BCIS）的重大进展通过为某些残疾人的生活提供了替代的交流方式，从而丰富了某些残疾人的生活。 但是，当前系统在很大程度上依赖于单峰技术，这些技术限制了它们的性能以及我们对正确解释多尺度神经功能的综合神经动力学的理解。 为了解决这个问题，已提议采用混合（多模式）BCIS，但试图通过简单组合（例如，来自两种神经影像模式的特征集的串联）来利用多种方式的互补益处，仅产生了增量的进步；融合多模式信号以有效，同时从感兴趣的多个信号中提取互补信息的可推广计算数据驱动的方法。 这项研究将探索一种用于混合非侵入性BCI系统的创新方法，该方法利用了互补的生理特征，该特征分别在基于图的数据融合框架的帮助下，分别使用EEG和FNIRS从电气和血液动力学信号获得。项目成果将包括新颖的信号处理管道，并为主流用户应用程序的实用BCI技术奠定基础。 除了该项目的潜在社会影响外，该团队还将专注于扩大参与STEM的参与，还将吸引来自K-12的学生到研究生层面。研究将涉及三个主要推力。 新的图理论多模式数据融合框架将被开发，以系统地捕获混合模式和用户意图的复杂拓扑特征，并同时调节感兴趣的电气和血液动力学响应。 由于多模式技术在时空分辨率和信息内容方面创建了固有的互补属性，因此该框架将旨在从隐藏在EEG中的复杂混合图案和FNIRS信号中的复杂混合图案中捕获相应的互补协同拓扑特征，以获得用户意图的高级抽象。 该框架将通过在现实世界中优化包含最高共同信息的参数和通道对非交流个体进行评估。 最后，将提出一个概念上新的基于混合子空间的过滤器，以最大程度地提高两类混合数据和增强分类性能之间的距离。该奖项反映了NSF的法定任务，并认为值得通过基金会的知识分子优点和更广泛的影响来通过评估来获得支持。

项目成果

A Graph-Based Nonlinear Dynamic Characterization of Motor Imagery Toward an Enhanced Hybrid BCI

• DOI: 10.1007/s12021-022-09595-2
• 发表时间: 2022-07
• 期刊: Neuroinformatics
• 影响因子: 3
• 作者: S. Hosni;S. B. Borgheai;J. McLinden;Shaotong Zhu;Xiaofei Huang;S. Ostadabbas;Y. Shahriari

A Graph-Based Feature Extraction Algorithm Towards a Robust Data Fusion Framework for Brain-Computer Interfaces

基于图的特征提取算法实现脑机接口的鲁棒数据融合框架

• 发表时间: 2021
• 期刊: 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society
• 作者: Zhu, Shaotong;Hosni, Sarah;McLinden, John;Borgheai, Bahram;Shahriari, Yalda;Ostadabbas, Sarah.
• 通讯作者: Ostadabbas, Sarah.

A Graph-Based Dynamical Characterization and Inference in Hybrid BCIs

混合 BCI 中基于图的动态表征和推理

• 发表时间: 2021
• 期刊: and Computers
• 作者: S. I Hosni, S. B.