Collaborative Research:CIF:Small: Acoustic-Optic Vision - Combining Ultrasonic Sonars with Visible Sensors for Robust Machine Perception

合作研究：CIF：Small：声光视觉 - 将超声波声纳与可见传感器相结合，实现强大的机器感知

• 批准号: 2326904
• 负责人: Adithya Pediredla
• 金额: $ 30万
• 依托单位: Dartmouth College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-01-01 至 2026-12-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326904&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Acoustic

Computer vision has primarily used optical information to understand the environment, from object detection to robotic localization and mapping. However, many animals in the natural world use multiple sensing modalities to explore their environments. In particular, hearing and acoustics have been used effectively with vision to enhance navigation in the wild. Dolphins and bats both use sonar echolocation to help triangulate their position in visually degraded environments such as underwater or at night. Inspired by their mammalian sensing systems, this project investigates the fusion of optical and ultrasonic sensors for enhanced computer vision and computational imaging. The team of researchers will develop novel hardware platforms for performing joint optical and acoustic sensing along with the corresponding graphics and vision algorithms to support simulation and analysis of this multimodal data. The resulting sensing platforms will be leveraged for applications in 3D scanning, material recognition, non-line-of-sight imaging, and robotic mapping. Further, the project features dissemination and education goals through incorporation of bio-inspired vision and acoustics into computer graphics and vision courses, undergraduate research, engagement with members of an industry-university consortium, and a middle-school outreach program.The project will focus on three main objectives. The first objective will be to design new rendering algorithms that can jointly synthesize optical and acoustical information in an environment. These algorithms will be made efficient by the importance sampling of optical rays and sound waves in an environment and will inform sensor models and sampling criteria to recover the environment information. The second objective will be to develop new sensing models that trade off sampling across optical and acoustic modalities. These systems will leverage multiple image sensors and acoustic transducers which will be prototyped in real hardware. The final objective will focus on the mutual benefit of combining vision and acoustics for robotics systems. Applications in 3D sensing, material identification, non-line-of-sight/occluded imaging, and acoustic-optic robotic navigation will be investigated. Research insights will be disseminated to the robotics, computer-vision, acoustics, and computational-imaging communities through scientific publications and presentations as well as open-source software and hardware designs for reproducibility.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机视觉主要使用光学信息来理解环境，从物体检测到机器人定位和绘图。然而，自然界中的许多动物使用多种传感方式来探索其环境。特别是，听觉和声学已与视觉有效地结合使用，以增强野外导航能力。海豚和蝙蝠都使用声纳回声定位来帮助在水下或夜间等视觉退化的环境中三角定位它们的位置。受哺乳动物传感系统的启发，该项目研究光学和超声波传感器的融合，以增强计算机视觉和计算成像。研究人员团队将开发新颖的硬件平台，用于执行联合光学和声学传感以及相应的图形和视觉算法，以支持这种多模态数据的模拟和分析。由此产生的传感平台将用于 3D 扫描、材料识别、非视距成像和机器人测绘等应用。此外，该项目还通过将仿生视觉和声学纳入计算机图形和视觉课程、本科生研究、与产学研联盟成员的合作以及中学推广计划来实现传播和教育目标。该项目将重点关注围绕三个主要目标。第一个目标是设计新的渲染算法，可以在环境中联合合成光学和声学信息。这些算法将通过对环境中的光线和声波进行重要性采样而变得高效，并将告知传感器模型和采样标准以恢复环境信息。第二个目标是开发新的传感模型，在光学和声学模式之间权衡采样。这些系统将利用多个图像传感器和声学传感器，这些传感器将在真实硬件中进行原型设计。最终目标将侧重于将机器人系统的视觉和声学相结合的互惠互利。将研究 3D 传感、材料识别、非视线/遮挡成像和声光机器人导航中的应用。研究见解将通过科学出版物和演示以及开源软件和硬件设计以实现可重复性，向机器人、计算机视觉、声学和计算成像社区传播。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。