Collaborative Research:CIF:Small:Acoustic-Optic Vision - Combining Ultrasonic Sonars with Visible Sensors for Robust Machine Perception

合作研究：CIF：Small：声光视觉 - 将超声波声纳与可见传感器相结合，实现强大的机器感知

• 批准号: 2326905
• 负责人: Suren Jayasuriya
• 金额: $ 30万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-01-01 至 2026-12-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326905&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Acoustic

Computer vision has primarily used optical information to understand the environment, from object detection to robotic localization and mapping. However, many animals in the natural world use multiple sensing modalities to explore their environments. In particular, hearing and acoustics have been used effectively with vision to enhance navigation in the wild. Dolphins and bats both use sonar echolocation to help triangulate their position in visually degraded environments such as underwater or at night. Inspired by their mammalian sensing systems, this project investigates the fusion of optical and ultrasonic sensors for enhanced computer vision and computational imaging. The team of researchers will develop novel hardware platforms for performing joint optical and acoustic sensing along with the corresponding graphics and vision algorithms to support simulation and analysis of this multimodal data. The resulting sensing platforms will be leveraged for applications in 3D scanning, material recognition, non-line-of-sight imaging, and robotic mapping. Further, the project features dissemination and education goals through incorporation of bio-inspired vision and acoustics into computer graphics and vision courses, undergraduate research, engagement with members of an industry-university consortium, and a middle-school outreach program.The project will focus on three main objectives. The first objective will be to design new rendering algorithms that can jointly synthesize optical and acoustical information in an environment. These algorithms will be made efficient by the importance sampling of optical rays and sound waves in an environment and will inform sensor models and sampling criteria to recover the environment information. The second objective will be to develop new sensing models that trade off sampling across optical and acoustic modalities. These systems will leverage multiple image sensors and acoustic transducers which will be prototyped in real hardware. The final objective will focus on the mutual benefit of combining vision and acoustics for robotics systems. Applications in 3D sensing, material identification, non-line-of-sight/occluded imaging, and acoustic-optic robotic navigation will be investigated. Research insights will be disseminated to the robotics, computer-vision, acoustics, and computational-imaging communities through scientific publications and presentations as well as open-source software and hardware designs for reproducibility.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机视觉主要使用光学信息来了解环境，从对象检测到机器人定位和映射。但是，自然世界中的许多动物都采用多种传感方式来探索其环境。特别是，听力和声学已被有效地与视觉相关，以增强野外的导航。海豚和蝙蝠都使用声纳回声定位来帮助三角剖分在视觉降低的环境中的位置，例如水下或晚上。受其哺乳动物传感系统的启发，该项目研究了光学和超声传感器的融合，以增强计算机视觉和计算成像。研究人员团队将开发新的硬件平台，以执行关节光学传感和相应的图形和视觉算法，以支持对该多模式数据的仿真和分析。最终的传感平台将用于3D扫描，材料识别，非视线成像和机器人映射中的应用。此外，该项目通过将生物启发的愿景和声学纳入计算机图形和视觉课程，本科研究，与行业大学联盟的成员的参与以及中学外展计划中的传播和教育目标。该项目将专注于三个主要目标。第一个目标是设计新的渲染算法，可以在环境中共同合成光学和声学信息。这些算法将通过在环境中的光射线和声波的重要性采样来提高这些算法，并将为传感器模型和采样标准提供信息以恢复环境信息。第二个目标是开发新的传感模型，以跨光学和声学方式进行取样。这些系统将利用多个图像传感器和声音传感器，这些传感器将在真实硬件中进行原型。最终目标将集中于将机器人系统的视觉和声学相结合的相互利益。将研究3D传感，材料识别，非视线/遮挡成像和声学机器人导航中的应用。研究见解将通过科学出版物和演示文稿，演示文稿以及开源软件和硬件设计，以进行重复可重复性。该奖项反映了NSF的法规任务，并被认为是通过基金会的知识优点和广泛的crietia crietia crietia crietia crietia crietia crietia crietia crister crcrietia crcrietia crcrietia crcri crcritia，这将通过科学出版物和演示文稿以及开源软件和硬件设计来分散。