Mechanisms of integrating haptic cues for VR-based interactive analytics of complex data

整合触觉线索以进行基于 VR 的复杂数据交互式分析的机制

• 批准号: RGPIN-2020-04052
• 负责人: Hu, Yaoping
• 金额: $ 2.04万
• 依托单位: University of Calgary
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752126
• 关键词: Mechanisms; integrating; haptic; cues; VR

PROBLEM: Many scientific/engineering applications yield increasingly complex data in multiple dimensions and variates. Although application-specific algorithms can extract certain ill-defined patterns of the data, the intricate characteristics of these patterns require experts (users) with various competencies to analyze them interactively and collaboratively (i.e., interactive analytics) for decision-making. High-end computers usually display the patterns as 3-dimensional objects in intertwisted shapes (local attributes) and interlinkages (global relationships). Routinely, the users interact with the objects via keyboards/mice while viewing them on screens. This practice deters the users from conducting interactive analytics efficiently, due to difficulties in relying solely on the sense of vision for decoding the interconnection between the local attributes and global relationships. A solution might employ the sense of touch to explore the local attributes, while maintaining the view of the global relationships. The solution involves the human ability of using both senses to interact with objects for achieving cognitive responses. Rendering visual and haptic (touch by force and vibrotactile) cues to stimulate both senses, a computer platform based on advanced Virtual Reality (VR) technologies has a potential for catalyzing interactive analytics. Impediments are however two major deficiencies in the current knowledgebase: (a) the lack of robust schemes for associating the haptic cues with the local attributes of the patterns, and (b) the unavailability of adequate models for integrating the cues to foster neuro-ergonomic interaction. OBJECTIVE: The long-term objective of this program is to address the deficiencies by creating computational mechanisms for integrating the haptic cues, hence catalysing VR-based interactive analytics. The short-term objectives of the program are to establish schemes of force and vibrotactile cues (Theme A) and to build models of integrating the cues (Theme B). IMPACT: It is challenging to invent cue-integration mechanisms in complying with behavioural and neural responses for fostering neuro-ergonomic interaction. Collaborating with national and international researchers, the program meets the challenge by achieving two innovative elements: (i) a methodology of creating cue schemes from pattern attributes, and (ii) an optimization of ensuring the cue-integration. The achievement signifies a knowledge gain to permit the use of haptic cues for facilitating interactive analytics, and a technical advancement to enable a viable VR-based platform for benefiting Canadian economy by promoting experts' actions in applications including: planning public safety (e.g., flood mitigation), boosting industrial competitiveness (e.g., reactor design), innovating trustable human-machine interfaces (e.g., human-robot defense systems), and inventing practical brain-machine interaction (e.g., object manipulation via neural signals).

问题：许多科学/工程应用会产生越来越复杂的多维度和多变量数据。尽管特定于应用程序的算法可以提取某些不明确的数据模式，但这些模式的复杂特征需要具有各种能力的专家（用户）以交互和协作方式对其进行分析（即交互式分析）以做出决策。高端计算机通常将模式显示为具有交织形状（局部属性）和互连（全局关系）的 3 维对象。通常，用户在屏幕上查看对象时通过键盘/鼠标与对象进行交互。这种做法阻碍了用户有效地进行交互式分析，因为仅依靠视觉来解码局部属性和全局关系之间的互连很困难。解决方案可能会利用触觉来探索局部属性，同时保持全局关系的视图。该解决方案涉及人类使用两种感官与物体交互以实现认知反应的能力。基于先进虚拟现实 (VR) 技术的计算机平台可渲染视觉和触觉（力触和振动触觉）线索来刺激两种感官，具有促进交互式分析的潜力。然而，当前知识库中存在两个主要缺陷：（a）缺乏将触觉提示与模式的局部属性相关联的稳健方案，以及（b）缺乏足够的模型来整合提示以促进神经工效学相互作用。目标：该计划的长期目标是通过创建集成触觉提示的计算机制来解决这些缺陷，从而促进基于 VR 的交互式分析。该计划的短期目标是建立力和振动触觉线索的方案（主题 A）并建立整合线索的模型（主题 B）。影响：发明线索整合机制来遵守行为和神经反应以促进神经人体工程学相互作用是具有挑战性的。该计划与国内和国际研究人员合作，通过实现两个创新要素来应对挑战：(i) 根据模式属性创建提示方案的方法，以及 (ii) 确保提示集成的优化。这一成就标志着知识的获取，允许使用触觉提示来促进交互式分析，以及技术的进步，以实现基于 VR 的可行平台，通过促进专家在应用中的行动来造福加拿大经济，包括：规划公共安全（例如，洪水）缓解），提高工业竞争力（例如，反应堆设计），创新可信赖的人机界面（例如，人机防御系统），并发明实用的脑机交互（例如，通过神经网络进行物体操作）信号）。