GI Video-to-3D: Gastrointestinal Tract 3D Modeling and Visualization for In-Vivo

GI 视频转 3D：体内胃肠道 3D 建模和可视化

• 批准号: 7534550
• 负责人: Jason Geng
• 金额: $ 28.15万
• 依托单位: XIGEN, LLC
• 依托单位国家: 美国
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-09-15 至 2009-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7534550
• 关键词: Address; Algorithms; Architecture; Area; Clinical; Color; Computational Science; Computer Assisted; Computer Security; Computer software; Computer-Assisted Diagnosis; Cystoscopes; Daily; Data Set; Deglutition; Devices; Diabetes Mellitus; Diagnosis; Dimensions; Disease; Disorder by Site; Endoscopes; Evaluation; Event; Freedom; Funding Opportunities; Gastrointestinal tract structure; Gastroscopes; Generations; Goals; Hour; Image; Image Analysis; Imagery; Institutes; Intervention; Kidney Diseases; Lead; Life; Location; Measurable; Measurement; Medical; Medical Surveillance; Modeling; Motion; National Institute of Diabetes and Digestive and Kidney Diseases; Numbers; Operative Surgical Procedures; Painless; Patients; Performance; Phase; Physicians; Positioning Attribute; Process; Public Health; Range; Rate; Research; Resolution; Robotics; Small Business Funding Mechanisms; Small Business Innovation Research Grant; Small Intestines; Software Design; Software Tools; Sports; Staging; Structure; System; Techniques; Technology; Testing; Texture; Three-Dimensional Image; Three-Dimensional Imaging; Time; Today; Transportation; Travel; United States National Institutes of Health; Visualization software; Wireless Technology; Work; base; bioimaging; capsule; design; experience; gastrointestinal; image processing; in vivo; innovation; instrument; interest; novel; pill; programs; reconstruction; response; sensor; simulation; size; success; three dimensional structure; three-dimensional modeling; two-dimensional; virtual reality

DESCRIPTION (provided by applicant): Recent advances in miniature camera technology have lead new generation endoscopic devices called capsule cameras [1~5] which use a swallowable pill-size miniature wireless video sensor to acquire and transmit video image sequences while traveling along the gastrointestinal(GI) tract. Although this revolutionary technology offers patients with painless examination experience, it however only provides two dimensional (2D) sequential video images that contains no three dimensional (3D) information of observed targets. Recognition and evaluation of pathological structures and the estimation of their spatial dimension can only be achieved by experience, making a diagnosis decision very subjective. For daily clinical usages, the unprecedented huge data set (more than 57,600 raw 2D images per exam) requires in average 45 minutes of physician's time to review. Locating the spatial position of a specific target with respect to patient's body is fairly difficult task due to lack of 3D information. The primary objective of this SBIR program is to develop a 3D image processing computational software platform, dubbed as the "Video-to-3DTM", which is able to (automatically) produce, for the first time, an integrated, patient-specific, and quantitatively measurable gastrointestinal (GI) tract 3D model based upon the 2D video sequence acquired by a capsule camera during an exam. The ability to perform quantitatively 3D measurement of pathological structures with calibrated color texture would make endoscopic diagnosis more objective and reproducible. Furthermore, an integrated 3D model of patient-specific GI tract and 3D fly-through visualization software capability would assist diagnosis and intervention planning, save physicians tremendous time in video reviewing, and provide accurate 3D intra-body localization of "target of interest" with respect to patient's body. Since a capsule camera moves freely inside the GI tract, its 6-degree-of-freedom motion is neither controllable nor accurately measurable. We therefore need to develop sophisticated computational efficient algorithms to obtain a very robust estimation of the camera motion from the uncalibrated image sequence, and then reconstruct 3D structure using the known camera motion and other 3D image processing techniques. The "Video-to-3DTM" Software Platform is designed to carry out the following tasks: (1) 3D Modeling: Inter-correlate over 57,600 images acquired by any capsule camera to reconstruct a high resolution patient-specific 3D model of GI tract; (2) 3D Visualization: Provide texture super-resolution and 3D fly-through capability for the 3D GI tract model to help physicians to visualize and diagnose quickly, interactively, accurately, and efficiently; (3) 3D Measurement: Perform quantitative 3D measurement of interested pathological structures; (4) 3D Localization: Determine accurate 3D intra-body location of targets within patient's body. (5) Computer-aided diagnosis: The ultimate goal of the proposed Video-to3D system is to be able to assist doctors to detect, classify, and identify certain targeted diseases. The proposed "Video-to-3DTM" adds one more dimension to the existing 2D capsule camera technology, literally and figuratively. Specific aims of Phase I effort to build the proposed Video-to-3DTM software include: Aim 1: Design software architecture for the Video-to-3D computational platform; Aim 2: Develop and optimize algorithms and software components; Aim 3: Perform extensive tests using simulation platform and images acquired by in-vivo capsule cameras; Aim 4: Assess Phase 1 system performance and prepare for Phase II work plan. PUBLIC HEALTH RELEVANCE: We propose to develop a 3D image processing computational software platform, dubbed as the "Video-to-3DTM", capable of producing (automatically) an integrated, patient-specific, and quantitatively measurable gastrointestinal (GI) tract 3D model based upon the 2D video sequence acquired by a capsule camera during an exam. The proposed Video-to-3DTM software platform blazes an entirely new trail in the in- vivo capsule camera technology by providing, for the first time, the unprecedented capability of 3D GI tract modeling, 3D fly-through visualization, 3D measurement of pathological structures and 3D intra-body localization of target area with respect to patient's body for diagnosis and intervention planning. The proposed "Video-to-3DTM" adds one more dimension to the existing capsule camera technology, literally and figuratively. Given the expanding needs in biomedical imaging research, the proposed software platform technology is ultimately generalizable, scalable, extensible, and interoperable. In addition to produce leapfrog performance advances for the revolutionary capsule camera technology and many other existing medical diagnosis instruments (endoscopes, gastroscope, cystoscope, colonscope, etc.), potential applications of 3D model reconstruction technology developed herein span a broad spectrum of many fields of applications, such as pipe inspection, turbine engine diagnosis, smart transportation systems, security and surveillance, vehicle navigation, mobile robotics, teleconferencing, virtual reality products, on-line distribution of live images, event and location viewing, urban modeling, entertainment, sports webcasts, and many others.

描述（由申请人提供）：微型相机技术的最新进展具有新一代的内窥镜设备，称为胶囊摄像机[1〜5]，该设备使用可吞咽的药丸大小的微型无线视频传感器在沿胃肠道旅行时获取和传输视频图像序列（gi）道。尽管这项革命性的技术为具有无痛经验的患者提供了，但它仅提供二维（2D）顺序的视频图像，这些视频图像不包含观察到的目标的三维（3D）信息。只有通过经验才能实现病理结构的识别和评估及其空间维度的估计，从而使诊断决策非常主观。对于每日临床用法，前所未有的巨大数据集（每次考试超过57,600张原始2D图像）平均需要45分钟的医生进行审查。由于缺乏3D信息，因此在患者身体方面找到特定目标的空间位置是相当艰巨的任务。该SBIR程序的主要目的是开发一个3D图像处理计算软件平台，该平台被称为“视频到3DTM”，该平台能够（自动）首次生产，这是一个集成，特定于患者的，并根据考试期间胶囊摄像头获得的2D视频序列进行定量可测量的胃肠道（GI）3D模型。具有校准色纹理的病理结构进行定量测量的3D测量的能力将使内窥镜诊断更加客观和可重复。此外，患者特异性的胃肠道和3D直通可视化软件能力的综合3D模型将有助于诊断和干预计划，在视频审查中节省大量时间，并提供准确的3D兴趣“目标”内部定位。尊重患者的身体。由于胶囊摄像头在胃肠道内部自由移动，因此其6度的自由运动既无法控制也无法准确测量。因此，我们需要开发复杂的计算有效算法，以从未校准的图像序列中对相机运动进行非常可靠的估计，然后使用已知的摄像头运动和其他3D图像处理技术重建3D结构。 “视频到3DTM”软件平台旨在执行以下任务：（1）3D建模：任何胶囊摄像头收购的57,600张图像超过57,600张图像，以重建高分辨率的GI Tract的高分辨率患者特异性3D模型； （2）3D可视化：为3D GI道模型提供纹理超分辨率和3D直飞能力，以帮助医生快速，互动，准确和有效地可视化和诊断； （3）3D测量：对感兴趣的病理结构进行定量3D测量； （4）3D定位：确定患者体内靶标的准确的3D体内内部位置。 （5）计算机辅助诊断：拟议的视频对3D系统的最终目标是能够帮助医生检测，分类和识别某些目标疾病。提出的“视频到3DTM”在字面上和形象上为现有的2D胶囊摄像头技术增加了一个维度。第一阶段努力构建所提出的视频到3DTM软件的特定目标包括：AIM 1：为视频到3D计算平台设计软件体系结构；目标2：开发和优化算法和软件组件； AIM 3：使用模拟平台和Vivo胶囊摄像头获得的图像进行广泛的测试； AIM 4：评估第1阶段系统性能并准备第二阶段工作计划。公共卫生相关性：我们建议开发一个3D图像处理计算软件平台，称为“视频到3DTM”，能够（自动）（自动）集成，特定于患者和定量测量的胃肠道（GI）3D模型（自动）。根据考试期间胶囊摄像头获得的2D视频序列。拟议的视频到3DTM软件平台通过首次提供3D GI道建模的前所未有的能力，3D蝇态可视化，3D测量病理结构，这是在体内胶囊摄像头技术中的全新步道。目标区域对患者的身体进行诊断和干预计划的3D体内定位。提出的“视频对3DTM”在字面上和形象上为现有胶囊摄像头技术增加了一个维度。鉴于生物医学成像研究的需求不断扩大，因此提出的软件平台技术最终是可推广的，可扩展的，可扩展的和可互操作的。除了为革命性胶囊摄像头技术和许多其他现有的医学诊断工具（内镜，胃镜，胃镜，膀胱镜，calonscope等）产生LeapFrog绩效进步外，此处开发的3D模型重建技术的潜在应用还跨越了许多领域的广泛领域。应用程序，例如管道检查，涡轮发动机诊断，智能运输系统，安全性和监视，车辆导航，移动机器人技术，电信，虚拟现实产品，实时图像的在线分布，活动和位置查看，城市建模，娱乐，娱乐，娱乐，体育，体育网络广播和许多其他。