CHS: Small: Collaborative Research: Sampling and Reconstruction for Computer Graphics Rendering and Imaging

CHS：小型：协作研究：计算机图形渲染和成像的采样和重建

• 批准号: 1451830
• 负责人: Ravi Ramamoorthi
• 金额: $ 25万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1451830&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Sampling

Sampling of high-dimensional signals is at the heart of graphical rendering and computational photography, but current approaches unfortunately still tend to be brute-force and require large numbers of samples, which is time-consuming and costly. In this project, which involves researchers at two institutions, the Principal Investigators will build on their prior work to develop a comprehensive theoretical, algorithmic and systems foundation for sampling and reconstruction in computer graphics rendering and imaging. A key goal is a unified sampling theory that considers the type of coherence in the visual signal (such as low rank, locally low rank, low frequency, sparsity) and the type of measurement (such as point samples in rendering or projection of generic patterns for light transport acquisition, or acquisition of full light field imagery). This will provide a unified framework for choosing the best sampling strategy, and for comparing different approaches. It will also enable the establishment of rigorous lower bounds and optimality results. The work has immediate connections to signal-processing, applied mathematics and photography, and will have broad impact in connecting these domains with computer graphics. The Principal Investigators will disseminate project outcomes in part by incorporating the findings into their online courses that have large enrolments. They will also make datasets and software available, and will work to include them in industrial applications by exploiting their strong ties with a number of high-tech companies. Physically-based rendering algorithms are now widespread in production, but photorealistic rendering is still inefficient since it involves the evaluation of a high-dimensional 4D-8D Monte Carlo integral for each pixel considering antialiasing, lens effects, motion blur, soft shadows and global illumination. Typically, each pixel is treated separately, with many samples needed for each integral dimension. Similar challenges arise in other areas of computer graphics, such as precomputed rendering (explicit tabulation of a 4D-8D light transport operator), light transport acquisition (measurement of high-dimensional 4D-8D functions like the BRDF or BSSRDF), and computational photography or imaging that acquires higher-dimensional 4D functions in consumer light field cameras. The traditional approach is to (pre)compute or measure the data by brute force, followed by compression. However, this incurs unacceptable costs given the size and dimensionality of current visual appearance datasets. In this work the Principal Investigators will leverage the sparsity in the continuous (rather than discrete Fourier) domain, coherence and structure of light transport to sample, reconstruct and integrate, reducing the amount of data needed by orders of magnitude, while developing new reconstruction schemes for computational imaging. Within rendering, the PIs will explore a novel method that combines motion blur, depth of field, and global illumination in a single algorithm for real-time rendering based on adaptive Monte Carlo sampling and filtering of different effects. A key challenge in such approaches is robust sampling of difficult paths; the Principal Investigators will address this issue with conservative adaptive sampling and Graduated Metropolis. Finally, new systems-level software will be developed that enables easy integration and implementation of light transport simulation methods for rendering and imaging.

高维信号采样是图形渲染和计算摄影的核心，但不幸的是，当前的方法仍然倾向于暴力，并且需要大量样本，这既耗时又昂贵。 在这个涉及两个机构的研究人员的项目中，主要研究人员将在他们之前的工作基础上，为计算机图形渲染和成像中的采样和重建开发全面的理论、算法和系统基础。 一个关键目标是统一采样理论，该理论考虑视觉信号中的一致性类型（例如低秩、局部低秩、低频、稀疏性）和测量类型（例如渲染或投影通用模式中的点样本）用于光传输采集，或采集全光场图像）。 这将为选择最佳采样策略和比较不同方法提供一个统一的框架。 它还将能够建立严格的下限和最优结果。 这项工作与信号处理、应用数学和摄影有着直接的联系，并将在这些领域与计算机图形学的连接方面产生广泛的影响。 首席研究员将通过将研究结果纳入其拥有大量注册人数的在线课程来传播项目成果。他们还将提供数据集和软件，并利用与许多高科技公司的牢固联系，努力将其纳入工业应用。基于物理的渲染算法现已在生产中广泛使用，但真实感渲染仍然效率低下，因为它涉及对每个像素进行高维 4D-8D 蒙特卡罗积分评估，考虑抗锯齿、镜头效果、运动模糊、软阴影和全局照明。 通常，每个像素都被单独处理，每个积分维度需要许多样本。 计算机图形学的其他领域也存在类似的挑战，例如预计算渲染（4D-8D 光传输算子的显式制表）、光传输采集（测量 BRDF 或 BSSRDF 等高维 4D-8D 函数）和计算摄影或在消费类光场相机中获取更高维 4D 功能的成像。 传统的方法是通过强力（预）计算或测量数据，然后进行压缩。 然而，考虑到当前视觉外观数据集的大小和维度，这会产生不可接受的成本。 在这项工作中，主要研究人员将利用连续（而不是离散傅立叶）域的稀疏性、光传输的相干性和结构来采样、重建和积分，将所需的数据量减少几个数量级，同时开发新的重建方案用于计算成像。 在渲染方面，PI 将探索一种新颖的方法，将运动模糊、景深和全局照明结合在单一算法中，以基于自适应蒙特卡罗采样和不同效果的过滤进行实时渲染。 这种方法的一个关键挑战是对困难路径进行稳健采样；首席研究员将通过保守的自适应抽样和分级大都市来解决这个问题。 最后，将开发新的系统级软件，以便轻松集成和实施用于渲染和成像的光传输模拟方法。