Collaborative Research: RI: Small: Motion Fields Understanding for Enhanced Long-Range Imaging

合作研究：RI：小型：增强远程成像的运动场理解

• 批准号: 2232298
• 负责人: Nianyi Li
• 金额: $ 30万
• 依托单位: Clemson University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2232298&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Motion

Data-driven computer vision approaches suffer from deteriorated performance when the input images are captured from long distance. For example, images from unmanned aerial vehicles (UAVs), satellites, and reconnaissance cameras lack stereo information causing 3D reconstruction and depth estimation to fail. Turbulence caused by air and water also causes light rays to deflect from their original path and introduces noticeable motion artifacts like blurriness and distortion. This project develops a generalizable motion field estimator using neural networks coupled with specific hardware settings to enhance computer vision tasks in long-range imaging. Successful development of such a motion field estimator can enable applications of computer vision systems at long distances and/or under turbulent environments including UAV navigation, object tracking and detection, and long-range monitoring. The project has broader impact in industrial applications which leverage such technologies. In addition, research results will be integrated into new course materials for physics-informed computer vision and computational photography classes. The project will provide training to underrepresented students and outreach to K-12 students throughout its duration. This project will develop computational solutions to decouple the entangled motion fields and use turbulence motion to enhance visual computing applications in long-range imaging. This research is motivated by the observation that turbulence-induced motion fields can provide depth and sub-pixel color information, which is crucial in restoring scenes with high-frequency details. To achieve this goal, the project will pursue three research thrusts: 1) neural field decoupling of object and turbulence motion; 2) reconstructing turbulence strength and flows from passive visual imagery; and 3) motion field guided intelligent foveation for long-range imaging. The first thrust will develop algorithms for estimating and recovering motion fields with both object and turbulence motion by investigating physics-based velocity fields. The second thrust will develop tractable quantitative turbulence motion models that can be applied to both air and water environments using deep neural networks. The third thrust will integrate the turbulence motion field into different visual computing pipelines to benefit long-range computer vision tasks. This project will collect a large motion field dataset with true turbulent parameters of different media types and turbulence strengths, which can facilitate the development of data-driven machine learning algorithms for long-range computer vision.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从长距离捕获输入图像时，数据驱动的计算机视觉方法的性能恶化。例如，来自无人驾驶汽车（无人机），卫星和侦察摄像机的图像缺乏立体信息信息，导致3D重建和深度估计失败。 由空气和水引起的湍流也会导致光线偏离其原始路径，并引入明显的运动伪影，例如模糊和失真。 该项目使用神经网络以及特定的硬件设置开发了可推广的运动场估计器，以增强长距离成像中的计算机视觉任务。这种运动场估计器的成功开发可以在长距离和/或在动荡的环境下启用计算机视觉系统的应用，包括无人机导航，对象跟踪和检测以及远程监控。该项目对利用此类技术的工业应用具有更大的影响。此外，研究结果将集成到物理知识的计算机视觉和计算摄影类的新课程材料中。该项目将为代表性不足的学生提供培训，并在K-12学生的整个过程中向K-12学生推广。该项目将开发计算解决方案，以使纠缠的运动场将其解散，并使用湍流运动来增强长距离成像中的视觉计算应用。这项研究是由观察到的，即湍流引起的运动场可以提供深度和子像素颜色信息，这对于恢复具有高频细节的场景至关重要。为了实现这一目标，该项目将追求三个研究推力：1）对象和湍流运动的神经场去耦； 2）重建湍流强度并从被动视觉图像中流动； 3）运动场引导智能效果进行远程成像。第一个推力将开发算法，用于通过研究基于物理的速度场来估计和恢复具有对象和湍流运动的运动场。第二个推力将开发可拖动的定量湍流运动模型，该模型可以使用深神经网络应用于空气和水环境。第三个推力将将湍流运动场整合到不同的视觉计算管道中，以使远程计算机视觉任务受益。该项目将收集一个具有不同媒体类型和湍流强度的真正动荡参数的大型运动现场数据集，这可以促进用于长期计算机视觉的数据驱动机器学习算法的开发。该奖项反映了NSF的法定任务，并通过使用该基金会的知识绩效和广泛的影响来评估NSF的法定任务，并被视为值得的支持。