Plausible multisensory modeling of 3D deformable objects for interactive applications

用于交互式应用的 3D 可变形物体的合理多感官建模

• 批准号: RGPIN-2014-04953
• 负责人: Cretu, AnaMaria
• 金额: $ 1.82万
• 依托单位: Carleton University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=648291
• 关键词: Plausible; multisensory; modeling; 3D; deformable

The introduction on the market of 3D enabled technologies, such as 3D displays, 3D glasses and of various means for interaction with virtual environments (gloves, tactile feedback equipment, etc.) places an increasing pressure on the research community to develop enabling technologies for 3D contents. To fully reach their potential, interactive environments need to go beyond the modeling of software generated objects by introducing plausible representations of 3D objects based on real measurements. The rich interaction behavior exhibited by real-world objects depends not only on how objects look, but also on their properties: how stiff they are, how their surface feels and sounds when touched, or how they deform on contact. The development of interactive models for the complex behavior of objects based on real measurements has not followed the rapid progress of visual and audio realism achieved in the entertainment and computer gaming industry. Rising up to this challenge, the long-term goal of this research program is to develop enhanced sensing and modeling techniques for applications that require higher realism by means of plausible 3D object models based on real multisensory measurements, along with their integration, representation and interpretation.**From the multitude of aspects involved in this long-term objective, this original research program focuses on three major aspects: (1) the acquisition and integration of visual, audio and tactile measurements into realistic, coherent 3D models; (2) the rendering of the complex behavior of such objects; and (3) the integration and evaluation of multisensory object models in the context of specific applications. More specifically, human visual and tactile sensing approaches, tactile and visual sampling strategies using various sensors, vision algorithms and computational intelligence techniques will be adapted and refined for use on real measurements. They will be advantageously combined for the formulation of novel object models based on multisensory visual, tactile and audio data. Original computational intelligence methods will be proposed to allow for an efficient use of various sources of information on the object behavior and for the seamless merge of multisensory models in interactive environments. In an initial stage, a dedicated interface will be implemented for the visualization and evaluation of the developed models. Next, the models will be validated in a Cave Automatic Virtual Environment (CAVE) where small imperfections become immediately perceptible. Models will finally be integrated into two practical applications, one for phobia treatment and the other one for safe model-based object manipulation of deformable objects. **The proposed program will contribute the theoretical and practical knowledge required to build the next generation of interactive applications that will rely on plausible 3D contents acquired from real measurements, instead of hard-coded, approximate computer-generated graphics models. Advancements in this area of research will have a significant impact on interactive applications that require higher accuracy and improved realism. These include, beyond the two applications above, interactive environments for training, interactive virtual prototyping for precision-sensitive industrial CAD-based processes, smart manufacturing as well as dexterous manipulation applications such as robotic assembly, tele-robotics, medical and assistive robotics. This research work will support the training of highly qualified specialists with cutting edge approaches and technologies, who are in demand for the strong multimedia, gaming, and virtual reality industry in Canada.

启用3D启用技术的介绍，例如3D显示器，3D眼镜以及与虚拟环境互动（手套，触觉反馈设备等）的各种手段，给研究社区带来了越来越多的压力，以开发3D的启用技术内容。为了充分发挥其潜力，交互式环境需要通过基于实际测量值引入3D对象的合理表示来超越软件生成的对象的建模。现实世界对象表现出的丰富相互作用行为不仅取决于对象的外观，还取决于它们的属性：它们的僵硬，触摸时的表面和声音的感觉和声音，或它们在接触时的变形方式。基于实际测量值的对象复杂行为的交互模型的开发并未遵循娱乐和计算机游戏行业中的视觉和音频现实主义的快速发展。在这一挑战中，该研究计划的长期目标是针对需要更高现实主义的应用程序来开发增强的感应和建模技术，这些应用程序通过基于真实的多感官测量值的合理3D对象模型以及它们的集成，表示和解释， 。 （2）此类对象的复杂行为的渲染； （3）在特定应用程序的上下文中对多感觉对象模型的集成和评估。更具体地说，使用各种传感器，视觉算法和计算智能技术的人类视觉和触觉传感方法，触觉和视觉采样策略将被调整和精制，以用于实际测量。它们将被有利地组合起来，以基于多感觉视觉，触觉和音频数据的新对象模型制定。将提出原始的计算智能方法，以有效利用有关对象行为的各种信息以及在交互式环境中多感官模型的无缝合并。在初始阶段，将实现专用界面，以实现开发模型的可视化和评估。接下来，将在洞穴自动虚拟环境（Cave）中对模型进行验证，在该环境（Cave）中，小型缺陷立即变得可见。最终将将模型集成到两个实际应用中，一种用于恐惧症治疗，另一个用于基于安全模型的对象操纵可变形物体。 **拟议的程序将贡献构建下一代交互式应用程序所需的理论和实践知识，这些应用程序将依赖于从实际测量值中获得的合理的3D内容，而不是硬编码的，近似的计算机生成的图形模型。该研究领域的进步将对需要更高准确性和改善现实主义的交互式应用产生重大影响。其中包括，除了上述两个应用程序外，还包括用于培训的交互式环境，交互式虚拟原型制作，用于基于精确的工业CAD的工艺，智能制造以及灵巧的操纵应用，例如机器人组装，电信，电视，医学，医学和辅助机器人技术。这项研究工作将通过尖端的方法和技术来支持对加拿大强大的多媒体，游戏和虚拟现实行业的培训。