RI: Small: Depth from Differential Defocus

RI：小：微分散焦的深度

• 批准号: 1718012
• 负责人: Todd Zickler
• 金额: $ 45万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1718012&HistoricalAwards=false
• 关键词: RI; Small; Depth; Differential; Defocus

This project will explore a new class of depth sensors. The new sensors operate by observing small changes in optical defocus through a single lens, and they require very small amounts of digital computation. The distinguishing feature of these sensors is that they can be much smaller and lower power than existing depth sensor technologies. By enabling depth sensing capabilities on smaller platforms, they help accelerate the creation of smart micro-scale systems and an effective Internet of things. Depth sensors produce two-dimensional images where each pixel's value is the distance to a scene point along a corresponding ray. A variety of these sensors exist, and they are already fueling advances in autonomous navigation, gesture-driven interfaces, robotics, and more.This research will develop sensors based on a new visual cue called differential defocus. Like the well-known passive depth cues of stereo and depth-from-defocus, this new cue avoids spending power on broadcasting light. But unlike the existing passive cues, it calculates depth using simple analytic expressions that are easy to compute. To establish differential defocus as a new way to sense depth, this project aims to discover a complete stack of knowledge, from mathematical foundations to algorithms and hardware prototypes. The mathematical foundations include a catalog of depth constraints that correspond to many forms of differential defocus, such as differential camera motion, sensor motion, change of focal length, and change of aperture. At the hardware level, the project will pursue both single-shot and multi-shot designs that incorporate deformable lenses and customized photosensors. Algorithmically, the project will explore methods for using back-propagation to fine-tune the parameters of depth computations. Going further, it will explore back-propagation into the optical dimension, in order to enable the optimization of optical and computational parameters together, in a synergistic manner.

该项目将探索新的深度传感器。新传感器通过单个镜头观察光学散焦的小变化来运行，并且需要少量的数字计算。这些传感器的区别特征是它们可以比现有的深度传感器技术更小，更低的功率。通过在较小平台上启用深度感测功能，它们有助于加速智能微型系统和有效的物体互联网的创建。深度传感器产生二维图像，其中每个像素的值是沿相应射线的场景点的距离。这些传感器都存在，并且它们已经在自动导航，手势驱动界面，机器人技术等方面取得了进步。这项研究将基于一种新的视觉提示来开发传感器，称为差分散焦。就像众所周知的被动深度提示和直接从框架中的深度线索一样，这种新提示避免了在广播光上花费的力量。但是，与现有的被动提示不同，它使用易于计算的简单分析表达式计算深度。为了建立差异化散焦作为感知深度的新方法，该项目旨在发现一堆知识，从数学基础到算法和硬件原型。数学基础包括与多种形式的差分散焦相对应的深度约束目录，例如降低摄像头运动，传感器运动，焦距的变化和光圈的变化。在硬件级别上，该项目将同时进行单次和多拍设计，这些设计结合了可变形的镜头和定制的光传感器。从算法上讲，该项目将探索使用后传播来微调深度计算参数的方法。再进一步，它将探索反向传播到光学维度中，以便以协同的方式将光学和计算参数的优化在一起。

项目成果

Focal Flow: Velocity and Depth from Differential Defocus Through Motion

• DOI: 10.1007/s11263-017-1051-5
• 发表时间: 2018-10
• 期刊: International Journal of Computer Vision
• 影响因子: 19.5
• 作者: Emma Alexander;Qi Guo;S. Koppal;S. Gortler;Todd E. Zickler

Focal Track: Depth and Accommodation with Oscillating Lens Deformation

焦点轨迹：具有振荡透镜变形的深度和调节

• DOI: 10.1109/iccv.2017.110
• 发表时间: 2017
• 期刊: IEEE International Conference on Computer Vision
• 作者: Guo, Qi;Alexander, Emma;Zickler, Todd
• 通讯作者: Zickler, Todd

Compact single-shot metalens depth sensors inspired by eyes of jumping spiders

• DOI: 10.1073/pnas.1912154116
• 发表时间: 2019-11-12
• 期刊: PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
• 影响因子: 11.1
• 作者: Guo, Qi;Shi, Zhujun;Zickler, Todd
• 通讯作者: Zickler, Todd