CCSS: Uncertainty-Aware Computational Imaging in the Wild: a Bayesian Deep Learning Approach in the Latent Space

CCSS：野外不确定性感知计算成像：潜在空间中的贝叶斯深度学习方法

• 批准号: 2348046
• 负责人: Xin Li
• 金额: $ 20万
• 依托单位: SUNY at Albany
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-11-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2348046&HistoricalAwards=false
• 关键词: CCSS; Uncertainty; Aware; Computational; Imaging

From microscopy and tomography to biometrics and surveillance, computational imaging (CI) has greatly expanded mankind’s vision capability in various scientific and engineering fields. In contrast to hardware-based solutions that require expensive optics, data-driven or computational approaches powered by deep learning have fueled the development of AI-enabled computational imaging systems. The emerging paradigm of Bayesian deep learning endows the next-generation CI systems more flexibility to handle various uncertainties in the wild – no matter whether they are related to the adversarial conditions of image acquisition (e.g., moving platform or bad weather) or unintentional mistakes made by humans (e.g., the mal-functioned main mirror of Hubble Space Telescope after the launch into the space). Developing uncertainty-aware CI systems have a wide range of impact on both scientific exploration (e.g., imaging at extreme scales such as microscopic and astronomical) and our daily lives (e.g., smartphone applications).This project aims at taking a Bayesian deep learning (BDL) approach to modeling uncertainty in real-world imaging scenarios. An important new insight brought about by this project is to unify the flow-based uncertainty kernel estimation (for likelihood modeling) with memory-based uncertainty image generation in latent space (for prior modeling). Under a fully Bayesian framework but using reparameterization to simplify the modeling process, the proposed BDL greatly facilitates both degradation learning and image reconstruction in the realistic scenario when uncertainty is inevitable. The proposed research consists of three tasks: 1) Flow-based Nonuniform Kernel Estimation in the Latent Space; 2) Memory-enhanced Deep Generative Models for Image Reconstruction; and 3) Real-world Applications in Deep Tissue Imaging such as Superresolution Microscopy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从显微镜和层析成像到生物识别技术和监视，计算成像（CI）在各种科学和工程领域的视觉能力都大大扩展。与基于硬件的解决方案相反，这些解决方案需要昂贵的光学，数据驱动的或由深度学习提供动力的计算方法，使AI-a-Spapentiment计算成像系统开发了。贝叶斯深度学习的新兴范式赋予下一代CI系统更灵活地处理野外各种不确定性 - 无论它们是否与图像获取的对抗性条件有关（例如，移动平台或恶劣的天气或恶劣的天气）还是人类所造成的无意识的错误（例如，人类造成的镜像镜像镜像远离镜像的空间，之后是型号的空间，之后又是启动了型号的空间。开发不确定性意识的CI系统对科学探索（例如，在极端尺度上进行成像，例如显微镜和天文学）以及我们的日常生活（例如，智能手机应用程序）。该项目旨在采取贝叶斯深度学习（BDL）方法，以实现现实世界成像的不确定的方法。该项目带来的一个重要的新见解是统一基于流动的不确定性内核估计（对于可能性建模），并在潜在空间中（用于先验建模）中基于内存的不确定性图像产生。在完全贝叶斯的框架下，但使用重新聚集来简化建模过程，当不确定的情况不确定性时，提出的BDL在现实情况下，在现实的情况下，拟议的BDL大大降解学习和图像重建。提出的研究包括三个任务：1）潜在空间中基于流动的非均匀核估计； 2）记忆增强图像重建的深层生成模型； 3）在深层组织成像中的现实应用（例如超分辨率显微镜）中的现实应用。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，通过评估来诚实地支持。