Scaling Volumetric Imaging, Analysis and Science Communication Using Immersive Virtual Reality

使用沉浸式虚拟现实扩展体积成像、分析和科学传播

• 批准号: 10604786
• 负责人: Gianfranco Doretto
• 金额: $ 79.89万
• 依托单位: ISTOVISR
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-01 至 2026-01-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10604786
• 关键词: 3-Dimensional; 4D Imaging; Academia; Acceleration; Active Learning; Adoption; BRAIN initiative; Biological; Brain; Brain imaging; Businesses; Clinical; Clinical Research; Cloud Computing; Communication; Computer software; Computers; Data; Data Set; Democracy; Depth Perception; Development; Digital Libraries; Educational Materials; Educational process of instructing; Electron Microscope; Ensure; Environment; Goals; Grant; Hour; Image; Image Analysis; Industry; Internet; Intuition; Knowledge; Language; Learning; Libraries; Licensing; Machine Learning; Manuscripts; Methods; Microscope; Modeling; Modernization; Multimedia; Narration; National Institute of Mental Health; Neurosciences Research; Pattern Recognition; Performance; Phase; Philosophy; Population; Process; Production; Publications; Reporting; Research; Resolution; Resources; Running; Sales; Scanning; Science; Scientist; Small Business Innovation Research Grant; Sound Localization; Structure; Students; System; Techniques; Technology; Testing; Three-Dimensional Image; Training; Training and Education; auditory processing; base; clinical imaging; clinical practice; cost; design; extended reality; field study; human-in-the-loop; improved; innovation; insight; large datasets; large scale data; learning strategy; lectures; machine learning algorithm; machine learning method; machine learning model; meetings; new technology; next generation; novel; peer; pre-clinical research; programs; prototype; scientific computing; stereoscopic; structural biology; success; technology development; tool; virtual reality; virtual reality environment

Over the past 15 years, new microscope technologies and methods for high throughput imaging have revolutionized structural biology by extending the resolution and scale of datasets in 3 dimensions. The resulting image volumes are more typically hundreds of GB to even tens of TB and for large volume electron microscope images of brain, can approach PB sizes. These file sizes pose challenges for image analysis, and communication of a representative set of raw data and quantification. Large files contain many structures, and require machine learning (ML) strategies in a context that permits error correction. Scientific communication requires tools for ready access to raw data, and more efficient methods to communicate the rapidly accumulating sets of scientific information. The rapidly accumulating digital library also affords a resource for teaching and training, which is largely untapped. We propose to leverage virtual reality (VR) to transform each of these challenges, capitalizing on natural abilities for stereoscopic vision and pattern recognition and, for scientific communication, teaching and training, auditory processing to process language and localize sounds. Based upon the tool base and direct volume rendering of large files that we have established in our VR software, called syGlass, we will first expand modern domain learning and so-called meta-learning techniques in the ML field to analyze images with few iterations from object counting to object tracking and tracing (Aim 1). Next, we will capitalize on new technologies for cloud rendering to significantly mitigate the hardware costs for adoption of syGlass (Aim 2). Finally, we will provide novel tools to efficiently generate narrated scientific presentations in VR for use in the lab setting, as manuscript publications, and for production of educational materials (Aim 3). The complexity of the brain offers a challenging testbed for teaching and training. In each of these Aims, we will introduce paradigm shifts in the analysis of the large data volumes, and communication of 3D and 4D data to colleagues and non-experts.

在过去的15年中，新的显微镜技术和高通量成像的方法 通过扩展数据集的分辨率和规模，彻底改变了结构生物学 方面。所得的图像量更常见的是数百GB至数十TB 对于大容量的电子显微镜图像，可以接近PB尺寸。这些文件 尺寸构成图像分析的挑战，以及一组代表性的原始挑战 数据和量化。大文件包含许多结构，需要机器学习（ML） 在允许纠正错误的情况下的策略。科学沟通需要工具 可以访问原始数据，以及更有效地传达迅速累积的方法 一组科学信息。迅速积累的数字图书馆还为 教学和培训，这在很大程度上尚未开发。 我们建议利用虚拟现实（VR）改变这些挑战，利用 关于立体视觉和模式识别的自然能力以及科学 沟通，教学和培训，听觉处理以处理语言和本地化 听起来。根据我们拥有的大型文件的工具库和直接音量渲染 在我们的VR软件（称为Syglass）中建立，我们将首先扩展现代领域学习和 ML字段中所谓的元学习技术，以分析图像很少的迭代图像 对象计算对象跟踪和跟踪（AIM 1）。接下来，我们将利用新的 云渲染的技术可显着减轻采用硬件的成本 Syglass（AIM 2）。最后，我们将提供新颖的工具来有效地生成叙述的科学 VR中的演示文稿用于实验室设置，作为手稿出版物以及生产 教育材料（目标3）。大脑的复杂性提供了一个具有挑战性的测试床 教学和培训。在每个目标中，我们都会在分析中引入范式转变 大数据量以及3D和4D数据与同事和非专家的通信。