I-Corps: Virtual 3D reconstruction of hollow organs from white light endoscopy

I-Corps：通过白光内窥镜对中空器官进行虚拟 3D 重建

• 批准号: 1602118
• 负责人: Audrey Bowden
• 金额: $ 5万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-11-01 至 2016-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1602118&HistoricalAwards=false
• 关键词: Corps; Virtual; 3D; reconstruction; hollow

White light endoscope (WLE) is a medical device that allows physicians to conduct real-time video explorations of the interior of organs to detect disease or guide surgery. Unfortunately, the WLE video data are too cumbersome in their native form to review post-treatment; hence, the information is often reduced to handwritten notes or a few still-image frames for inclusion in medical records. The loss of this visually rich information limits the ability of WLE to inform clinical decisions about treatment and surgery. In particular, diseases like bladder cancer, which holds the unfortunate distinction as the 4th most common cancer in men and the highest treatment cost per patient-lifetime of all cancers, would benefit from novel ways to review WLE video data to facilitate early detection of tumors and to better track changes in the bladder wall of patients likely to experience recurrence ( 50%). The goal of this project is to develop new technology to produce 3D visualizations of the interior of hollow organs such as the bladder from WLE videos. The availability of such technology will provide physicians with new tools to make better informed decisions about treatment and surgery, as well as provide researchers with new technology to enable novel studies on disease progression, ultimately leading to better health outcomes and lower treatment costs for diseases like cancer.The proposed novel algorithm applies state-of-the-art techniques in computer vision to the problem of 3D reconstruction of the shape and surface appearance of hollow organs. A key innovation in the proposed approach is that the algorithm is suitable to reconstruct a full, 3D model of an organ from endoscopic video captured with standard clinical hardware and requires only minor modifications to the standard clinical workflow. The ability to create these reconstructions from standard equipment and workflows arises from careful design decisions regarding (1) the protocol for endoscopic video collection, (2) necessary image pre-processing steps and (3) the particular combination of state-of-the-art techniques developed in the computer vision community into an end-to-end pipeline unique for our application. In brief, the overall algorithm involves the following steps: down-sample raw image data, process selected frames, determine camera poses, extrapolate the organ surface as a mesh and apply image-based texture to the finalized mesh. The team's initial application is for reconstructing urinary bladder using standard rigid cystoscopy data; to date the team has validated the ability to perform 3D reconstructions using standard clinical cystoscopy videos of 30 human patients. As the proposed method can powerfully augment the visual medical record of internal organ appearance, it is broadly applicable to endoscopy and represents a significant advance in monitoring the appearance of a patient's organ over time, as may be well suited for applications such as cancer surveillance.

白光内窥镜（WLE）是一种医疗设备，允许医生对器官内部进行实时视频探查，以检测疾病或指导手术。不幸的是，WLE 视频数据的原始形式太麻烦，无法在处理后进行查看；因此，信息通常被简化为手写笔记或一些静态图像帧以包含在医疗记录中。这种丰富的视觉信息的丢失限制了 WLE 为有关治疗和手术的临床决策提供信息的能力。特别是像膀胱癌这样的疾病，不幸的是，它被列为男性第四大常见癌症，也是所有癌症中患者一生治疗成本最高的疾病，它将受益于审查 WLE 视频数据的新方法，以促进肿瘤的早期发现并更好地跟踪可能复发的患者膀胱壁的变化（50%）。该项目的目标是开发新技术，通过 WLE 视频生成中空器官（例如膀胱）内部的 3D 可视化。此类技术的可用性将为医生提供新工具，以便就治疗和手术做出更明智的决策，并为研究人员提供新技术，以便对疾病进展进行新的研究，最终带来更好的健康结果并降低诸如此类疾病的治疗成本。所提出的新算法将计算机视觉中最先进的技术应用于中空器官形状和表面外观的 3D 重建问题。所提出方法的一个关键创新在于，该算法适合从使用标准临床硬件捕获的内窥镜视频重建器官的完整 3D 模型，并且只需要对标准临床工作流程进行少量修改。从标准设备和工作流程创建这些重建的能力源于对以下方面的仔细设计决策：(1) 内窥镜视频采集协议、(2) 必要的图像预处理步骤和 (3) 最新技术的特定组合计算机视觉社区中开发的艺术技术成为我们应用程序独有的端到端管道。简而言之，整个算法涉及以下步骤：对原始图像数据进行下采样、处理选定的帧、确定相机姿势、将器官表面外推为网格并将基于图像的纹理应用于最终的网格。该团队的最初应用是使用标准硬性膀胱镜检查数据重建膀胱；迄今为止，该团队已经验证了使用 30 名人类患者的标准临床膀胱镜检查视频进行 3D 重建的能力。由于所提出的方法可以有力地增强内脏器官外观的视觉医疗记录，因此它广泛适用于内窥镜检查，并且代表了随着时间的推移监测患者器官外观的显着进步，可能非常适合癌症监测等应用。