Full 3D Immersion: Capture and Visualization

全 3D 沉浸式体验：捕捉和可视化

• 批准号: RGPIN-2015-06812
• 负责人: Roy, Sébastien
• 金额: $ 1.75万
• 依托单位: Université de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=679741
• 关键词: Full; 3D; Immersion; Capture; Visualization

This grant proposal is in the field of computer vision. Its primary focus is on developing new 3D immersion technologies, from the perspective of capturing reality accurately as well as visualization (large immersive projections, single virtual headset). The goals are simple. First, we intend to provide highly accurate, fast, and robust methods for capturing reality immersively (large field of view) and in 3D, without any constraint on the motion of camera or the scene observed. Second, we intend to provide highly efficient methods for large scale visualization of immersive 3D content (i.e. dome multi-projection, or other large surfaces), and for small scale visualization, such as virtual headsets (i.e. oculus rift).******3D Immersion is becoming more prevalent everyday, especially in the arts, entertainment, and video games. This is due mostly to recent advances in virtual headset technology such as the Oculus rift and to new interest toward 3D dome projection, where the screen is not only filling the observers fields of view, but also provides stereo images, so depth can be perceived accurately in all directions. Content creation for 3D immersion is mostly performed with 3D computer graphics, where it is easy to make stereo-immersive rendering. However, the accurate depiction of reality requires real world filming and has remained a challenge. Traditional "small field of view" stereo capture can not be used directly for immersion because it provides a single point of view while immersive capture requires capturing from all possible point of views simultaneously. The single person immersive 3D experience is also evolving rapidly in the form of new high performance virtual reality headsets. Software for viewing content has matured, especially with regards to 3D real-time graphics and video games. However, for immersive video, the large resolutions involved and geometry corrections (including parallax) still constitute a challenge. Our research program intends to explore 3D immersion, from the perspective of content creation and content presentation.******We will develop new cameras for 3D spherical capture of reality, so it can be observed somewhere else with accurate depth perception in any direction, and in real-time.******We will also develop better multi-projection, to simplify and automate its deployment while making it more perceptually accurate immersion with depth.***Our long term goal is the "ultimate immersive 3D" experience, where the observer not only controls his gaze, but also his location. This new freedom for the observer will have a profound impact on the creation of 3D immersive content and make it much more challenging, thereby opening new avenues of research. Overall, if this research program is successful, we will have contributed to achieving the ultimate goal of 3D immersion: giving full freedom of movement to the observer, giving him a true felling of omnipresence.

该拨款提案属于计算机视觉领域。其主要重点是从准确捕捉现实以及可视化（大型沉浸式投影、单个虚拟耳机）的角度开发新的 3D 沉浸式技术。目标很简单。首先，我们打算提供高度准确、快速且稳健的方法，以沉浸式（大视场）和 3D 方式捕捉现实，而不会对相机的运动或观察到的场景产生任何限制。其次，我们打算为沉浸式 3D 内容的大规模可视化（即圆顶多投影或其他大型表面）和小规模可视化（例如虚拟耳机（即 oculus rift））提供高效的方法。**** **3D 沉浸式体验每天都变得越来越普遍，尤其是在艺术、娱乐和视频游戏领域。这主要是由于 Oculus Rift 等虚拟耳机技术的最新进展以及对 3D 圆顶投影的新兴趣，其中屏幕不仅填充观察者的视野，而且还提供立体图像，因此可以准确地感知深度四面八方。 3D 沉浸式内容创建主要通过 3D 计算机图形来执行，很容易进行立体沉浸式渲染。然而，准确描述现实需要现实世界的拍摄，这仍然是一个挑战。传统的“小视场”立体捕捉不能直接用于沉浸，因为它提供单一视点，而沉浸式捕捉需要同时从所有可能的视点进行捕捉。单人沉浸式 3D 体验也在以新型高性能虚拟现实耳机的形式迅速发展。用于查看内容的软件已经成熟，特别是在 3D 实时图形和视频游戏方面。然而，对于沉浸式视频来说，涉及的大分辨率和几何校正（包括视差）仍然构成挑战。我们的研究计划旨在从内容创建和内容呈现的角度探索 3D 沉浸感。******我们将开发用于现实的 3D 球形捕捉的新相机，以便可以在任何其他地方以准确的深度感知来观察它。 ******我们还将开发更好的多投影，以简化和自动化其部署，同时使其在深度上更加准确的沉浸感。***我们的长期目标是“最终身临其境的 3D”体验，观察者不仅控制他的目光，还控制他的位置。观察者的这种新的自由将对 3D 沉浸式内容的创作产生深远的影响，并使其更具挑战性，从而开辟新的研究途径。总的来说，如果这个研究计划成功，我们将为实现 3D 沉浸式的最终目标做出贡献：为观察者提供充分的移动自由，让他真正感受到无所不在。