HCC: Small: Super Reality -- Immersive Visual Interfaces with No Line-of-Sight Restriction

HCC：小：超现实——无视线限制的沉浸式视觉界面

基本信息

批准号：
2219842
负责人：
Voicu Popescu
金额：
$ 25万
依托单位：
Purdue University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2219842&HistoricalAwards=false
关键词：
HCC Small Super Reality Immersive

项目摘要

The project develops Super Reality, a new class of human-computer visual interfaces designed for efficient and effective 3D dataset exploration. Super Reality (SR) interfaces build upon conventional virtual reality interfaces, preserving their benefits of immersion, intuitiveness, and depth cues, while radically increasing the information bandwidth of the images shown to the user. SR interfaces let the user see more of the dataset from the current location by removing the line-of-sight restriction of conventional virtual reality interfaces. The user sees not only the regions of the dataset that are visible from the current viewpoint, but also regions that are hidden behind occluders. The user can preview an occluded dataset region from the current position, without having to navigate around occluders, thereby avoiding the wasted navigation when the occluded region proves to be of no interest. The user can compare distant dataset regions directly, in the same image, without having to rely on the memory of dataset regions seen earlier. With a SR interface, the user explores the dataset in parallel, from multiple viewpoints, but without the burden of a complex visualization interface based on a matrix of conventional images. Through the leap-forward increase of 3D dataset exploration efficiency and effectiveness, this project will lead to a foundational level innovation influencing the domains where data visualization is used. This SR interface has the potential to minimize user disorientation and cybersickness, which are triggered by mismatched user motion and user visual perception in the traditional virtual reality interfaces. The project will be carried out by involving a diverse group of undergraduate students in research.The goal of the project is the design and development of Super Reality (SR) interfaces that will remove the line-of-sight restriction through a generalization of the computational camera that projects the 3D dataset onto the 2D image plane. The linear rays of conventional cameras are replaced with curved rays that are routed around occluders to connect the user to dataset regions of interest. Super Reality interfaces essentially dispatch a team of virtual subordinate users that each image the dataset from their own viewpoint and present the user with an integration of the data captured from the multiple viewpoints. These integrated images are constructed to meet multiple design requirements: (a) image continuity, by bounding the difference between trajectories of neighboring rays; (b) image non-redundancy, by avoiding ray intersections; (c) adaptive image resolution, by routing more rays towards dataset regions of higher complexity or importance; (d) in-place disocclusion, by projecting the disoccluded region where the user would see it in the absence of the occluder; (e) minimal image distortion, by limiting the curvature of the non-linear rays; fast rendering, by designing the generalized cameras with a fast projection operation. The SR interfaces will be designed to render the part of these integrated images close to the user with the conventional, first-person view. This way, the part of the frame that corresponds to nearby geometry responds to the user’s view changes as the user expects, anchoring the user to avoid disorientation and cybersickness. Also, SR interface limits the number of degrees of freedom the user needs to manipulate in order to deploy and retract the desired disocclusion effects. The Super Reality interfaces are iteratively refined and validated in controlled user studies based on objective and subjective metrics that quantify the degree to which the design requirements are met, as well as the efficiency and effectiveness of 3D dataset exploration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目开发了超级现实（Super Reality），这是一种新型人机视觉界面，专为高效且有效的 3D 数据集探索而设计，超级现实（SR）界面建立在传统虚拟现实界面的基础上，保留了其沉浸感、直观性和深度提示的优点，同时。 SR 界面消除了传统虚拟现实界面的视线限制，从根本上增加了向用户显示的图像的信息带宽，让用户可以看到更多的数据集。从当前视点可见的数据集区域，以及隐藏在遮挡物后面的区域，用户可以从当前位置预览遮挡的数据集区域，而无需在遮挡物周围导航，从而避免遮挡时的浪费导航。事实证明，用户可以在同一图像中直接比较远处的数据集区域，而不必依赖先前看到的数据集区域的内存，通过 SR 接口，用户可以从多个并行地探索数据集。观点，但没有基于传统图像矩阵的复杂可视化界面的负担，通过 3D 数据集探索效率和有效性的飞跃提高，该项目将带来影响数据可视化使用领域的基础级创新。该 SR 界面有可能最大限度地减少用户迷失方向和晕动症，这是由传统虚拟现实界面中不匹配的用户运动和用户视觉感知引发的。该项目将通过让不同的本科生群体参与研究来进行。项目目标是超现实 (SR) 接口的设计和开发，它将通过将 3D 数据集投影到 2D 图像平面上的计算相机的泛化来消除视线限制。传统相机的线性光线被弯曲取代。围绕遮挡物传送的光线将用户连接到感兴趣的数据集区域，本质上是派遣一组虚拟的从属用户，每个用户从自己的角度对数据集进行成像，并向用户展示从其中捕获的数据的集成。这些集成图像的构造是为了满足多种设计要求：（a）通过限制相邻光线轨迹之间的差异来实现图像连续性；（b）通过避免光线交叉来实现图像非冗余；，通过将更多光线路由到更高复杂性或重要性的数据集区域；(d) 就地去除遮挡，通过将去除遮挡的区域投影到用户在没有遮挡物的情况下会看到的区域；(e) 最小图像；扭曲，通过限制非线性光线的曲率；快速渲染，通过设计具有快速投影操作的通用相机，SR接口将首先被设计为将这些集成图像的部分渲染为接近用户。这样，与附近几何图形相对应的框架部分就会按照用户的预期响应用户的视图变化，从而锚定用户以避免迷失方向和晕眩感。此外，SR 界面还限制了用户的视角变化。用户需要操纵的自由度，以便部署和收回所需的遮挡效果。超级现实界面在受控用户研究中根据客观和主观指标进行迭代完善和验证，这些指标量化了设计要求的满足程度。作为 3D 数据集探索的效率和有效性。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。