CCSS: Uncertainty-Aware Computational Imaging in the Wild: a Bayesian Deep Learning Approach in the Latent Space

CCSS：野外不确定性感知计算成像：潜在空间中的贝叶斯深度学习方法

• 批准号: 2348046
• 负责人: Xin Li
• 金额: $ 20万
• 依托单位: SUNY at Albany
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-11-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2348046&HistoricalAwards=false
• 关键词: CCSS; Uncertainty; Aware; Computational; Imaging

From microscopy and tomography to biometrics and surveillance, computational imaging (CI) has greatly expanded mankind’s vision capability in various scientific and engineering fields. In contrast to hardware-based solutions that require expensive optics, data-driven or computational approaches powered by deep learning have fueled the development of AI-enabled computational imaging systems. The emerging paradigm of Bayesian deep learning endows the next-generation CI systems more flexibility to handle various uncertainties in the wild – no matter whether they are related to the adversarial conditions of image acquisition (e.g., moving platform or bad weather) or unintentional mistakes made by humans (e.g., the mal-functioned main mirror of Hubble Space Telescope after the launch into the space). Developing uncertainty-aware CI systems have a wide range of impact on both scientific exploration (e.g., imaging at extreme scales such as microscopic and astronomical) and our daily lives (e.g., smartphone applications).This project aims at taking a Bayesian deep learning (BDL) approach to modeling uncertainty in real-world imaging scenarios. An important new insight brought about by this project is to unify the flow-based uncertainty kernel estimation (for likelihood modeling) with memory-based uncertainty image generation in latent space (for prior modeling). Under a fully Bayesian framework but using reparameterization to simplify the modeling process, the proposed BDL greatly facilitates both degradation learning and image reconstruction in the realistic scenario when uncertainty is inevitable. The proposed research consists of three tasks: 1) Flow-based Nonuniform Kernel Estimation in the Latent Space; 2) Memory-enhanced Deep Generative Models for Image Reconstruction; and 3) Real-world Applications in Deep Tissue Imaging such as Superresolution Microscopy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从显微镜和断层扫描到生物识别和监控，计算成像 (CI) 极大地扩展了人类在各个科学和工程领域的视觉能力。推动了人工智能计算成像系统的发展，贝叶斯深度学习的新兴范式使下一代 CI 系统能够更加灵活地处理野外的各种不确定性——无论它们是否与对抗条件有关。图像采集（例如，移动平台或恶劣天气）或人类无意的错误（例如，哈勃太空望远镜发射到太空后主镜出现故障）会产生广泛的影响。科学探索（例如，微观和天文等极端尺度的成像）和我们的日常生活（例如，智能手机应用程序）。该项目旨在采用贝叶斯深度学习该项目带来的一个重要的新见解是将基于流的不确定性核估计（用于似然建模）与潜在空间中基于内存的不确定性图像生成相结合。在完全贝叶斯框架下，但使用重新参数化来简化建模过程，当不确定性不可避免时，所提出的 BDL 极大地促进了现实场景中的退化学习和图像重建。基于非均匀核潜在空间估计；2）用于图像重建的记忆增强深度生成模型；3）深层组织成像的实际应用，例如超分辨率显微镜。该奖项反映了 NSF 的法定使命，并通过使用评估被认为值得支持基金会的智力价值和更广泛的影响审查标准。