CGV: Small: Collaborative Research: Sparse Reconstruction and Frequency Analysis for Computer Graphics Rendering and Imaging

CGV：小型：协作研究：计算机图形渲染和成像的稀疏重建和频率分析

• 批准号: 1115242
• 负责人: Ravi Ramamoorthi
• 金额: $ 25万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-10-01 至 2015-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1115242&HistoricalAwards=false
• 关键词: CGV; Small; Collaborative; Research; Sparse

A broad range of problems in computer graphics rendering, appearance acquisition, and imaging, involve sampling, reconstruction, and integration of high-dimensional (4D-8D) signals. Real-time rendering of glossy materials and intricate lighting effects like caustics, for example, can require pre-computing the response of the scene to different light and viewing directions, which is often a 6D dataset. Similarly, image-based appearance acquisition of facial details, car paint, or glazed wood requires us to take images from different light and view directions. Even offline rendering of visual effects like motion blur from a fast-moving car, or depth of field, involves high-dimensional sampling across time and lens aperture. The same problems are also common in computational imaging applications such as light field cameras. While the PIs and others have made significant progress in subsequent analysis and compact representation for some of these problems, the initial full dataset must almost always still be acquired or computed by brute force which is prohibitively expensive, taking hours to days of computation and acquisition time, as well as being a challenge for memory usage and storage.The PIs' goal in this project is to make fundamental contributions that enable dramatically sparser sampling and reconstruction of these signals, before the full dataset is acquired or simulated. The key idea is to exploit the structure of the data that often lies in lower-frequency, sparse, or low-dimensional spaces. Their recent collaboration on a Fourier analysis of motion blur has shown that the frequency spectrum of dynamic scenes is sheared into a narrow wedge in the space-time domain. This enables novel sheared (not axis-aligned) filters and a sparse sampling. The PIs will build upon these preliminary results to develop a unified framework for frequency analysis and sparse data reconstruction of visual appearance in computer graphics. To these ends, they will first lay the theoretical foundations, including a novel frequency analysis of Monte Carlo integration and 5D space-time analysis of light fields. They will then develop efficient practical algorithms for a variety of problem domains, including sparse reconstruction of light transport matrices for relighting, sheared sampling and denoising for offline shadow rendering, time-coherent compressive sampling for appearance acquisition, and new approaches to computational photography and imaging.Broader Impacts: From a theoretical perspective, this project will develop a fundamental signal-processing analysis of light transport and appearance and imaging datasets, which will provide the foundation for further work not just in computer graphics but in signal-processing, computer vision, and image analysis as well. Project outcomes will apply to diverse sets of problems and will lead to transformative advances across the spectrum of rendering and imaging applications. The PIs will leverage existing collaborations with industry to transition the new technologies to practical production use. Outreach to K-12 students and the public will be enabled by a new science popularization blog that will leverage the public's excitement for advances in digital photography to introduce novel technical concepts, as well as by events such as the Computer Science Education Day for high school students at UC-Berkeley. The new algorithms and datasets resulting from this work will be made available to the research community; moreover, imaging algorithms will be released in open-source format to work with consumer digital and cell-phone cameras.

计算机图形渲染、外观采集和成像中的广泛问题涉及高维 (4D-8D) 信号的采样、重建和集成。 例如，光泽材质和焦散等复杂照明效果的实时渲染可能需要预先计算场景对不同光线和观察方向的响应，这通常是 6D 数据集。 同样，基于图像的面部细节、汽车油漆或釉面木材的外观采集需要我们从不同的光线和观察方向拍摄图像。 即使是视觉效果的离线渲染，例如快速移动的汽车的运动模糊或景深，也涉及跨时间和镜头光圈的高维采样。 同样的问题在光场相机等计算成像应用中也很常见。 虽然 PI 和其他人在后续分析和对其中一些问题的紧凑表示方面取得了重大进展，但初始完整数据集几乎仍然必须通过暴力获取或计算，这是非常昂贵的，需要数小时到数天的计算和获取时间以及对内存使用和存储的挑战。PI 在该项目中的目标是做出基本贡献，在获取或模拟完整数据集之前实现这些信号的显着稀疏采样和重建。 关键思想是利用通常位于低频、稀疏或低维空间中的数据结构。 他们最近对运动模糊的傅立叶分析的合作表明，动态场景的频谱在时空域中被剪切成窄楔形。 这使得新颖的剪切（非轴对齐）滤波器和稀疏采样成为可能。 PI 将在这些初步结果的基础上开发一个统一的框架，用于计算机图形学中视觉外观的频率分析和稀疏数据重建。 为此，他们将首先奠定理论基础，包括蒙特卡罗积分的新颖频率分析和光场的 5D 时空分析。 然后，他们将为各种问题领域开发高效的实用算法，包括用于重新照明的光传输矩阵的稀疏重建、用于离线阴影渲染的剪切采样和去噪、用于外观采集的时间相干压缩采样以及计算摄影和成像的新方法更广泛的影响：从理论角度来看，该项目将开发光传输、外观和成像数据集的基本信号处理分析，这不仅将为计算机图形学领域的进一步工作奠定基础，而且还将为信号处理、计算机视觉、和图像分析也是如此。 项目成果将适用于各种不同的问题，并将带来渲染和成像应用领域的变革性进步。 PI 将利用与业界的现有合作，将新技术转化为实际生产用途。 将通过一个新的科普博客以及高中计算机科学教育日等活动，向 K-12 学生和公众进行宣传，该博客将利用公众对数码摄影进步的兴奋来介绍新颖的技术概念加州大学伯克利分校的学生。 这项工作产生的新算法和数据集将提供给研究界；此外，成像算法将以开源格式发布，以便与消费数码相机和手机相机配合使用。