CAREER: Fast Foveation: Bringing Active Vision into the Camera

职业：快速注视点：将主动视觉带入相机

• 批准号: 1942444
• 负责人: Sanjeev Koppal
• 金额: $ 51.93万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942444&HistoricalAwards=false
• 关键词: CAREER; Fast; Foveation; Bringing; Active

The prevalence of foveation, and the wide variety of it in the living world, makes it very clear that this is an effective visual design strategy. This project is about copying foveation, by building fast cameras that can optically concentrate sensing resources onto areas of interest in the world around them. Doing this can improve sensing performance for computer vision-enabled intelligent systems. On resource-constrained platforms, such as robots or spacecraft, adaptively sensing only on areas of interest improves efficiency. This project will create capability that enables a variety of sensing applications. Throughout the project timeline, research outcomes will be integrated in the investigator's hardware/software bridging courses, focused on fundamental procedures such as camera calibration. In addition, a program called LensLearning will be started, to spread foveating camera concepts beyond the lab. LensLearning includes impacting high-school students through special University of Florida programs with hands-on projects. It also enables the training of one high-school student and one undergraduate senior every summer through this project's timeline, by working with the University of Florida's associated programs, with the goal of giving opportunities to underrepresented minorities in foveated camera research.Although the idea of artificial foveation has been explored with slow, mechanical means of motion, in this project the foveating cameras and accompanying algorithms will be much faster because they exploit newly available, next generation micro-mechanical optics that can quickly and adaptively change the camera resolution. The first phase of this project involves building the fast foveating camera test-bed and characterizing the fundamental limits of fast foveation for dynamic scenes through an optical model that considers modulation speed, camera field-of-view, noise, motion and long-range effects. The second phase involves demonstrating tracking advantages in dynamic scenes with variants of the fast foveation setup, such as co-located systems and arrays of foveating cameras. Evaluations in simulation will be done using widely available datasets by comparing processing power and imaging efficiency. Real evaluation will also be done on the test bed, resulting in the release of a novel foveated dataset of dynamic scenes of everyday objects. In the last phase, the developed systems and algorithms will be used to demonstrate extreme imaging applications by combining both large baselines and co-located multimodal systems, showing capabilities such as glasses-free eye-tracking, imaging in dark environments and fast face imaging for robotics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Foveation的普遍性及其在生活世界中的各种各样的流行率非常清楚地表明，这是一种有效的视觉设计策略。该项目是关于复制foveation的，该项目是通过构建可以将传感资源光学地集中到周围世界感兴趣领域的快速摄像机来进行的。这样做可以改善具有计算机视觉的智能系统的传感性能。在资源受限的平台（例如机器人或航天器）上，仅适应感兴趣的领域可提高效率。该项目将创建能够实现各种传感应用程序的功能。在整个项目时间表中，研究成果将集成到研究者的硬件/软件桥接课程中，重点是诸如摄像机校准之类的基本程序。此外，将启动一个名为“晶状体学习”的程序，以将相机概念传播到实验室之外。镜头学习包括通过佛罗里达大学特殊大学计划通过动手项目来影响高中生。它还可以通过与佛罗里达大学的相关计划合作，通过该项目的时间表对一名高中生和一名高中生和一名本科大四学生进行培训，目的是提供机会，以使人工繁荣的概念在人工散发的概念中，尽管在这个项目中，他们的动作量很长，并且随着人们的范围而探索，因为他们的机械手段和陪同人员都可以探索，因为他们的动作范围是福特的，因为他们的范围是福特的范围。利用新的下一代微型机械光学器件，可以快速并适应性地改变相机分辨率。该项目的第一阶段涉及构建快速的摄像机测试床，并通过视力模型来表征动态场景的快速效力的基本限制，该光学模型考虑了调制速度，视野，摄像头，噪声，运动，运动和远距离效果。第二阶段涉及在动态场景中展示带有快速凹陷设置的变体的跟踪优势，例如共同关联的系统和foveating摄像机的数组。通过比较处理能力和成像效率，将使用广泛可用的数据集进行仿真评估。实际评估还将在测试床上进行，从而释放了新颖的日常物体动态场景数据集。在最后阶段，开发的系统和算法将通过结合大型基准和共同确定的多模式系统来证明极端的成像应用，显示诸如无眼镜的能力，黑暗环境中的成像，在黑暗环境中进行成像以及快速面对面的成像，用于机器人的奖励，这些奖项反映了NSF的合法任务和良好的依据。 标准。

项目成果

Energy-Efficient Adaptive 3D Sensing

节能的自适应 3D 传感

• 发表时间: 2023
• 期刊: IEEE Conference on Computer Vision and Pattern Recognition
• 作者: Tilmon, Brevin;Sun, Zhanghao;Koppal, Sanjeev J.;Wu, Yicheng;Evangelidis, Georgios;Zahreddine, Ramzi;Krishnan, Gurunandan;Ma, Sizhuo;Wang, Jian
• 通讯作者: Wang, Jian

Fast Foveating Cameras for Dense Adaptive Resolution

快速注视点相机，实现密集自适应分辨率

• DOI: 10.1109/tpami.2021.3071588
• 发表时间: 2021
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Tilmon, Brevin;Jain, Eakta;Ferrari, Silvia;Koppal, Sanjeev Jagannatha
• 通讯作者: Koppal, Sanjeev Jagannatha