RI: Small: Collaborative Research: Dynamic Light Transport Acquisition and Applications to Computational Illumination

RI：小型：合作研究：动态光传输采集及其在计算照明中的应用

• 批准号: 1909192
• 负责人: Suren Jayasuriya
• 金额: $ 25万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909192&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Dynamic

Cameras that view a dynamic scene typically capture interactions of moving objects with light. Computer vision algorithms can use these measurements to infer properties of these objects, such as depth, motion and appearance. However, there is a subtler, richer visual back-story that occurs as an object moves in a scene, and usually these effects are ignored in traditional algorithms, sometimes causing errors. This project studies all the interactions of light with dynamic scenes, which we term as dynamic light transport, and the goal is to understand and recover effects such as multiple reflections and scattering as objects move in a scene. The innovations of the project include new computational cameras and projectors to capture light transport for dynamic scenes, and to explore new physics-based and data-driven algorithms to exploit this information for improved computer vision and graphics applications. The project further seeks to include broadening access to computing education and research through curriculum material and capstone experiences which emphasize the intersection of light transport and digital media as well as outreach to middle and high school students in summer programs to discover imaging and optics applications.This research focuses on designing new light transport acquisition frameworks to capture dynamic scenes, characterization of dynamic light transport properties including sparsity and low-rank, and algorithms to exploit this information for computer vision applications. In particular, the project focuses on three main objectives. The first is design of an MEMs-based optical scanner coupled with high frame rate cameras to capture the full set of light transport paths at extremely fast timescales. The second contribution is new algorithms for adaptive light transport sampling using both physics-based and data-driven priors for light transport interpolation via generalized light transport flow. Finally, the project will provide applications of dynamic light transport for 3D scanning of deformable, moving, and specular objects. These innovations are evaluated in an integrated testbed via the optical scanner and the collection of a dataset of dynamic light transport for real-world scenes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

查看动态场景的相机通常会用光捕获移动对象的交互。计算机视觉算法可以使用这些测量值来推断这些对象的性质，例如深度，运动和外观。但是，随着对象在场景中移动时，有一个微妙，更丰富的视觉背景故事，通常在传统算法中忽略了这些效果，有时会导致错误。该项目研究了光线与动态场景的所有相互作用，我们将其称为动态光传输，目标是理解和恢复效果，例如随着对象在场景中移动时的多种反射和散射。该项目的创新包括新的计算机和投影仪，以捕获动态场景的光运输，并探索基于物理和数据驱动的新算法，以利用此信息来改善计算机视觉和图形应用程序。 The project further seeks to include broadening access to computing education and research through curriculum material and capstone experiences which emphasize the intersection of light transport and digital media as well as outreach to middle and high school students in summer programs to discover imaging and optics applications.This research focuses on designing new light transport acquisition frameworks to capture dynamic scenes, characterization of dynamic light transport properties including sparsity and low-rank, and algorithms to exploit this计算机视觉应用的信息。特别是，该项目着重于三个主要目标。 首先是设计基于MEMS的光学扫描仪以及高框架速率摄像头的设计，以捕获非常快速的时间表的整套轻型传输路径。 第二个贡献是使用基于物理和数据驱动的先验进行自适应光传输采样的新算法，该算法通过广义光传输流量进行轻型传输插值。 最后，该项目将提供动态光传输的应用，以进行3D扫描，移动和镜面对象的3D扫描。这些创新通过光学扫描仪在集成测试床上进行了评估，并收集了现实世界中动态光传输数据集。该奖项反映了NSF的法定任务，并被认为值得通过基金会的知识分子优点和更广泛的影响标准通过评估来进行评估。

项目成果

Event-based dual photography for transparent scene reconstruction

基于事件的双重摄影，用于透明场景重建

• DOI: 10.1364/ol.483047
• 发表时间: 2023
• 期刊: Optics Letters
• 影响因子: 3.6
• 作者: Liu, Xiaomeng;Rego, Joshua D.;Jayasuriya, Suren;Koppal, Sanjeev J.
• 通讯作者: Koppal, Sanjeev J.

Differentiable Programming for Hyperspectral Unmixing using a Physics-based Dispersion Model

• DOI: 10.1007/978-3-030-58583-9_39
• 发表时间: 2020-07
• 作者: J. Janiczek;Parth Thaker;Gautam Dasarathy;C. Edwards;P. Christensen;Suren Jayasuriya

Dense Lissajous sampling and interpolation for dynamic light-transport

用于动态光传输的密集利萨如采样和插值

• DOI: 10.1364/oe.425061
• 发表时间: 2021
• 期刊: Optics Express
• 影响因子: 3.8
• 作者: Liu, Xiaomeng;Henderson, Kristofer;Rego, Joshua;Jayasuriya, Suren;Koppal, Sanjeev
• 通讯作者: Koppal, Sanjeev

Programmable Non-Epipolar Indirect Light Transport: Capture and Analysis

可编程非极间接光传输：捕获和分析

• DOI: 10.1109/tvcg.2019.2946812
• 发表时间: 2021
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: Kubo, Hiroyuki;Jayasuriya, Suren;Iwaguchi, Takafumi;Funatomi, Takuya;Mukaigawa, Yasuhiro;Narasimhan, Srinivasa G.
• 通讯作者: Narasimhan, Srinivasa G.

Temporally Consistent Relighting for Portrait Videos

• DOI: 10.1109/wacvw54805.2022.00079
• 发表时间: 2022-01
• 期刊: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)
• 作者: Sreenithy Chandran;Yannick Hold-Geoffroy;Kalyan Sunkavalli;Zhixin Shu;Suren Jayasuriya