Collaborative Research: Computational Photo-Scatterography: Unraveling Scattered Photons for Bio-Imaging

合作研究：计算光散射术：解开生物成像的散射光子

• 批准号: 1730574
• 负责人: Ashutosh Sabharwal
• 金额: $ 501.34万
• 依托单位: William Marsh Rice University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-03-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1730574&HistoricalAwards=false
• 关键词: Collaborative; Research; Computational; Photo; Scatterography

Much of the success of today's healthcare is due to rapid advances in our ability to collect and analyze high-resolution data about the human body. However, current methods to achieve cellular resolution are invasive (e.g., blood test or tissue biopsy), and non-invasive imaging modalities do not achieve cellular resolution. The principal goal of this Expeditions project is to develop computational imaging systems for non-invasive bio-imaging, deep beneath the skin, and at cellular-level resolutions. This project has the potential to fundamentally impact healthcare and medicine, by enabling live views of cross sections of human anatomy, simply by pointing a camera at any part of the body. This would put individual users at the center of their healthcare experience and make them true partners in their healthcare delivery. The health imaging devices that result from this project will act as an important pillar in the personalized medicine revolution. This research expedition also holds the potential to launch new healthcare paradigms for chronic disease management, pediatrics, low-resource healthcare, and disaster medical care. Beyond healthcare, making progress on the problem of cellular-scale deep-tissue imaging using light will push the frontiers of the fundamental problem of inverse scattering, which impacts numerous areas of science and engineering. The order of magnitude advances made in inverse scattering and imaging through scattering media will have significant cross-cutting applications in diverse areas such as basic science, consumer imaging, automotive navigation, robotics, surveillance, atmospheric science, and material science. Finally, projects with a single, easy-to-appreciate, and high-impact goal have the potential to inspire the next generation of scientists, attract diverse set of students driven by humanitarian and social causes, and become a platform for inclusion and innovation.The overarching goal of this project is to develop, test, and validate new computational imaging systems, to non-invasively image below the skin at tunable depths, in highly portable form-factors such as wearables or point-of-care devices. The main challenge is that light scatters as it travels through the human body, and in this process, the spatial information from different points within the body gets mixed up. A new concept, Computational Photo-Scatterography (CPS), is being applied in this project in order to computationally unravel the scattered photons in an imaging system, and allow creation of sharp images and accurate inferences. Recognizing that the brute-force complexity of unraveling scattered photons is prohibitively high, the project uses a computational co-design framework that leverages advances by team members from multiple domains: programmable illumination and optics, image sensors, machine learning, inverse graphics, and hybrid analog-digital computing. The project will use machine learning (ML) instead of physics-based de-scattering to speed up the solution of the underlying inverse problem. A combination of physics-based inverse graphics algorithms, and ML algorithms combining deep learning and generative modeling will be used to estimate tissue scattering parameters - motion due to blood flow induces time-variation in tissue parameters, which makes solving the inverse scattering problem more difficult. The project will use ML to create fast but approximate estimators, which will serve as accelerators for inverse scattering. The development of new sensors, able to capture the data necessary to reconstruct the structure of the tissue deep below the skin, constitutes the most important contribution of the project. These systems and algorithms will have the potential to break the current resolution limits of noninvasive bio-imaging by nearly two orders of magnitude, enabling cellular-level imaging at depths far beyond currently possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当今医疗保健的成功很大程度上归功于我们收集和分析人体高分辨率数据的能力的快速进步。然而，目前实现细胞分辨率的方法是侵入性的（例如血液测试或组织活检），而非侵入性成像方式无法实现细胞分辨率。该探险项目的主要目标是开发计算成像系统，用于皮下深处的细胞级分辨率的非侵入性生物成像。该项目有可能从根本上影响医疗保健和医学，只需将相机对准身体的任何部位即可实时查看人体解剖学的横截面。这将使个人用户成为其医疗保健体验的中心，并使他们成为医疗保健服务中真正的合作伙伴。该项目产生的健康成像设备将成为个性化医疗革命的重要支柱。这项研究还有可能为慢性病管理、儿科、低资源医疗保健和灾难医疗保健推出新的医疗保健范式。除了医疗保健之外，利用光在细胞级深层组织成像问题上取得进展将推动逆散射这一基本问题的前沿，从而影响科学和工程的众多领域。逆散射和通过散射介质成像所取得的数量级进步将在基础科学、消费者成像、汽车导航、机器人、监视、大气科学和材料科学等不同领域产生重大的交叉应用。最后，具有单一、易于理解和高影响力目标的项目有可能激励下一代科学家，吸引受人道主义和社会事业驱动的多元化学生，并成为包容和创新的平台。该项目的总体目标是开发、测试和验证新的计算成像系统，以可调节深度、高度便携的形式（例如可穿戴设备或护理点设备）对皮肤下进行非侵入性成像。主要挑战是光穿过人体时会发生散射，在这个过程中，来自体内不同点的空间信息会混合在一起。该项目中应用了一个新概念，即计算光散射术（CPS），以便通过计算方式解开成像系统中的散射光子，并创建清晰的图像和准确的推理。认识到解开散射光子的强力复杂性非常高，该项目使用了计算协同设计框架，该框架利用了来自多个领域的团队成员的进步：可编程照明和光学、图像传感器、机器学习、逆向图形和混合模拟数字计算。 该项目将使用机器学习（ML）而不是基于物理的去散射来加速底层逆问题的解决。基于物理的逆图形算法以及深度学习和生成建模相结合的机器学习算法将用于估计组织散射参数——血流引起的运动会引起组织参数的时间变化，这使得解决逆散射问题变得更加困难。该项目将使用机器学习来创建快速但近似的估计器，它将充当逆散射的加速器。新传感器的开发能够捕获重建皮肤深处组织结构所需的数据，是该项目最重要的贡献。这些系统和算法将有可能突破当前无创生物成像的分辨率限制近两个数量级，从而实现远远超出目前可能的深度的细胞级成像。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。

项目成果

Deep Learning Techniques for Inverse Problems in Imaging

• DOI: 10.1109/jsait.2020.2991563
• 发表时间: 2020-05-01
• 期刊: IEEE Journal on Selected Areas in Information Theory
• 作者: Ongie, Gregory;Jalal, Ajil;Willett, Rebecca
• 通讯作者: Willett, Rebecca

Snapshot polarimetric diffuse-specular separation

快照偏振漫反射镜面分离

• DOI: 10.1364/oe.460984
• 发表时间: 2022
• 期刊: Optics Express
• 影响因子: 3.8
• 作者: Dave, Akshat;Hold-Geoffroy, Yannick;Hašan, Miloš;Sunkavalli, Kalyan;Veeraraghavan, Ashok
• 通讯作者: Veeraraghavan, Ashok

Thermal Image Processing via Physics-Inspired Deep Networks

• DOI: 10.1109/iccvw54120.2021.00451
• 发表时间: 2021-08
• 期刊: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
• 作者: Vishwanath Saragadam;Akshat Dave;A. Veeraraghavan;Richard Baraniuk

Evaluating generative networks using Gaussian mixtures of image features

• DOI: 10.1109/wacv56688.2023.00036
• 发表时间: 2021-10
• 期刊: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• 作者: L. Luzi;Carlos Ortiz Marrero;Nile Wynar;Richard Baraniuk;Michael J. Henry

MINER: Multiscale Implicit Neural Representation

MINER：多尺度隐式神经表示

• 发表时间: 2022
• 期刊: European Conference on Computer Vision (ECCV
• 作者: Saragadam, Vishwanath;Tan, Jasper;Balakrishnan, Guha;Baraniuk, Richard G.;Veeraraghavan, Ashok
• 通讯作者: Veeraraghavan, Ashok