CHS: Small: Collaborative Research: EEG-Guided Electrical Stimulation for Immersive Virtual Reality

CHS：小型：合作研究：脑电图引导的沉浸式虚拟现实电刺激

• 批准号: 1717654
• 负责人: Murat Akcakaya
• 金额: $ 35.7万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717654&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; EEG

Spatial presence, in Virtual Reality (VR) terminology, refers to the perception (or illusion) of being physically present in a simulated environment. VR strives to create interactive environments that provide experiences of spatial presence through accurate delivery and perception of multimodal sensory stimuli. Research in VR spans fields ranging from neuroscience and medicine to gaming. While the computing and gaming industries have generated tremendous advances in hardware and software for graphics processing and 3D display technologies, VR systems still lack capabilities for providing users with haptic feedback (a sense of touch), which is crucial for generating truly immersive, real-world experiences. It is known that an increase in the feeling of spatial presence manifests itself in the form of increased brain activity. This research aims to achieve the control of haptic sensory stimulation adaptively, based on the changes in brain activity associated with perceptual responses elicited by sensory stimulation in VR environments. Project outcomes will include novel scientific discoveries and engineering enhancements that will make significant contributions to other areas of interest, such as prosthetic limbs, augmented reality, and telepresence applications. The project will help train a new generation of engineers skilled in addressing multidisciplinary challenges, while through outreach activities STEM careers will be promoted at the K-12 level.The research objective is to identify and analyze brain activity associated with the increased feeling of haptic spatial presence elicited by electro-tactile stimulation and measured through EEG, and to investigate closed-loop techniques to control electro-tactile stimulation for enhanced haptic presence in VR environments. Specifically, the project will: (1) develop an electrical haptic stimulation framework; (2) design analysis techniques to identify markers of haptic inputs in EEG; (3) establish control policies for adaptive electrical stimulation; and (4) evaluate and refine EEG-guided adaptive stimuli control framework in VR environments. In particular, the proposition to actuate haptic feedback through electrical stimulation is novel, while formulating design principles for model-based optimal EEG-guided closed-loop haptic feedback for immersive spatial presence is transformative. Additional innovative propositions to advance adaptive control under uncertainty and psychophysical investigations are unique; these present a potentially game-changing opportunity for VR system development and perhaps for general human-computer interaction.

在虚拟现实 (VR) 术语中，空间临场感是指在模拟环境中物理存在的感知（或幻觉）。 VR 致力于创建交互式环境，通过准确传递和感知多模态感官刺激来提供空间临场体验。 VR 研究涵盖从神经科学、医学到游戏等领域。 尽管计算和游戏行业在图形处理和 3D 显示技术的硬件和软件方面取得了巨大进步，但 VR 系统仍然缺乏为用户提供触觉反馈（触觉）的能力，而这对于生成真正身临其境的真实体验至关重要。世界经验。 众所周知，空间存在感的增加表现为大脑活动增加的形式。 本研究旨在根据 VR 环境中感官刺激引起的感知反应相关的大脑活动变化，实现对触觉感官刺激的自适应控制。 项目成果将包括新颖的科学发现和工程改进，这将为其他感兴趣的领域做出重大贡献，例如假肢、增强现实和远程呈现应用。 该项目将帮助培训能够应对多学科挑战的新一代工程师，同时通过外展活动，STEM 职业将在 K-12 级别得到提升。研究目标是识别和分析与触觉空间感增强相关的大脑活动通过电触觉刺激引发并通过脑电图测量存在感，并研究控制电触觉刺激的闭环技术，以增强 VR 环境中的触觉存在感。 具体来说，该项目将：（1）开发电触觉刺激框架； (2) 设计分析技术来识别脑电图中触觉输入的标记； (3)建立适应性电刺激的控制策略； (4)评估和完善VR环境中脑电图引导的自适应刺激控制框架。 特别是，通过电刺激驱动触觉反馈的主张是新颖的，而为沉浸式空间存在的基于模型的最佳脑电图引导闭环触觉反馈制定设计原则是变革性的。 在不确定性和心理物理学研究下推进适应性控制的其他创新主张是独一无二的；这些为 VR 系统开发乃至一般人机交互提供了潜在的改变游戏规则的机会。

项目成果

Optimal Query Selection Using Multi-Armed Bandits

使用多臂老虎机的最佳查询选择

• DOI: 10.1109/lsp.2018.2878066
• 发表时间: 2018-12
• 期刊: IEEE Signal Processing Letters
• 影响因子: 3.9
• 作者: Kocanaogullari, Aziz;Marghi, Yeganeh M.;Akcakaya, Murat;Erdogmus, Deniz
• 通讯作者: Erdogmus, Deniz

EEG-based trial-by-trial texture classification during active touch

主动触摸期间基于脑电图的逐次试验纹理分类

• DOI: 10.1038/s41598-020-77439-7
• 发表时间: 2020-11-27
• 期刊: Scientific Reports
• 影响因子: 4.6
• 作者: Eldeeb S;Weber D;Ting J;Demir A;Erdogmus D;Akcakaya M
• 通讯作者: Akcakaya M

EEG-based texture roughness classification in active tactile exploration with invariant representation learning networks

使用不变表示学习网络进行主动触觉探索中基于脑电图的纹理粗糙度分类

• DOI: 10.1016/j.bspc.2021.102507
• 发表时间: 2021-05
• 期刊: Biomedical Signal Processing and Control
• 影响因子: 5.1
• 作者: Özdenizci, Ozan;Eldeeb, Safaa;Demir, Andaç;Erdoğmuş, Deniz;Akçakaya, Murat
• 通讯作者: Akçakaya, Murat

An Event-Driven AR-Process Model for EEG-Based BCIs With Rapid Trial Sequences

具有快速试验序列的基于 EEG 的 BCI 的事件驱动 AR 过程模型

• DOI: 10.1109/tnsre.2019.2903840
• 发表时间: 2019-05
• 期刊: IEEE Transactions on Neural Systems and Rehabilitation Engineering
• 影响因子: 4.9
• 作者: Gonzalez;Marghi, Yeganeh M.;Azari, Bahar;Akcakaya, Murat;Erdogmus, Deniz
• 通讯作者: Erdogmus, Deniz

On Analysis of Active Querying for Recursive State Estimation

递归状态估计的主动查询分析

• DOI: 10.1109/lsp.2018.2823271
• 发表时间: 2018-06
• 期刊: IEEE Signal Processing Letters
• 影响因子: 3.9
• 作者: Kocanaogullari, Aziz;Erdogmus, Deniz;Akcakaya, Murat
• 通讯作者: Akcakaya, Murat