Scaling Volumetric Imaging, Analysis and Science Communication Using Immersive Virtual Reality

使用沉浸式虚拟现实扩展体积成像、分析和科学传播

基本信息

批准号：
10604786
负责人：
Gianfranco Doretto
金额：
$ 79.89万
依托单位：
ISTOVISR
依托单位国家：
美国
项目类别：
财政年份：
2020
资助国家：
美国
起止时间：
2020-05-01 至 2026-01-31
项目状态：
未结题

来源：
https://reporter.nih.gov/project-details/10604786
关键词：
3-Dimensional 4D Imaging Academia Acceleration Active Learning Adoption BRAIN initiative Biological Brain Brain imaging Businesses Clinical Clinical Research Cloud Computing Communication Computer software Computers Data Data Set Democracy Depth Perception Development Digital Libraries Educational Materials Educational process of instructing Electron Microscope Ensure Environment Goals Grant Hour Image Image Analysis Industry Internet Intuition Knowledge Language Learning Libraries Licensing Machine Learning Manuscripts Methods Microscope Modeling Modernization Multimedia Narration National Institute of Mental Health Neurosciences Research Pattern Recognition Performance Phase Philosophy Population Process Production Publications Reporting Research Resolution Resources Running Sales Scanning Science Scientist Small Business Innovation Research Grant Sound Localization Structure Students System Techniques Technology Testing Three-Dimensional Image Training Training and Education auditory processing base clinical imaging clinical practice cost design extended reality field study human-in-the-loop improved innovation insight large datasets large scale data learning strategy lectures machine learning algorithm machine learning method machine learning model meetings new technology next generation novel peer pre-clinical research programs prototype scientific computing stereoscopic structural biology success technology development tool virtual reality virtual reality environment

项目摘要

Over the past 15 years, new microscope technologies and methods for high throughput imaging have revolutionized structural biology by extending the resolution and scale of datasets in 3 dimensions. The resulting image volumes are more typically hundreds of GB to even tens of TB and for large volume electron microscope images of brain, can approach PB sizes. These file sizes pose challenges for image analysis, and communication of a representative set of raw data and quantification. Large files contain many structures, and require machine learning (ML) strategies in a context that permits error correction. Scientific communication requires tools for ready access to raw data, and more efficient methods to communicate the rapidly accumulating sets of scientific information. The rapidly accumulating digital library also affords a resource for teaching and training, which is largely untapped. We propose to leverage virtual reality (VR) to transform each of these challenges, capitalizing on natural abilities for stereoscopic vision and pattern recognition and, for scientific communication, teaching and training, auditory processing to process language and localize sounds. Based upon the tool base and direct volume rendering of large files that we have established in our VR software, called syGlass, we will first expand modern domain learning and so-called meta-learning techniques in the ML field to analyze images with few iterations from object counting to object tracking and tracing (Aim 1). Next, we will capitalize on new technologies for cloud rendering to significantly mitigate the hardware costs for adoption of syGlass (Aim 2). Finally, we will provide novel tools to efficiently generate narrated scientific presentations in VR for use in the lab setting, as manuscript publications, and for production of educational materials (Aim 3). The complexity of the brain offers a challenging testbed for teaching and training. In each of these Aims, we will introduce paradigm shifts in the analysis of the large data volumes, and communication of 3D and 4D data to colleagues and non-experts.

过去 15 年，用于高通量成像的新显微镜技术和方法通过扩展数据集的分辨率和规模，彻底改变了结构生物学 3 方面。生成的图像容量通常为数百 GB 甚至数十 TB 对于大体积的大脑电子显微镜图像，可以接近 PB 大小。这些文件尺寸对图像分析和代表性原始数据集的通信提出了挑战数据和量化。大文件包含许多结构，需要机器学习 (ML) 允许错误纠正的上下文中的策略。科学传播需要工具随时访问原始数据，以及更有效的方法来传达快速积累的数据科学信息集。快速积累的数字图书馆也为教学和培训，这在很大程度上尚未开发。我们建议利用虚拟现实 (VR) 来转变这些挑战，充分利用立体视觉和模式识别的自然能力以及科学沟通、教学和培训、听觉处理以处理语言和本地化声音。基于我们拥有的工具库和大文件的直接体积渲染在我们的 VR 软件 syGlass 中建立的，我们将首先扩展现代领域学习和机器学习领域中所谓的元学习技术，可以通过很少的迭代来分析图像从对象计数到对象跟踪和追踪（目标 1）。接下来，我们将利用新云渲染技术可显着降低采用云渲染的硬件成本 syGlass（目标 2）。最后，我们将提供新颖的工具来有效地生成叙述性科学成果 VR 演示文稿可在实验室环境中使用、作为手稿出版物以及用于制作教育材料（目标 3）。大脑的复杂性提供了一个具有挑战性的测试平台教学和培训。在每个目标中，我们将在分析中引入范式转变大数据量以及与同事和非专家进行 3D 和 4D 数据的交流。