Evaluation and optimization of NWB neurophysiology software and data in the cloud

NWB 神经生理学软件和云数据的评估和优化

• 批准号: 10827688
• 负责人: Benjamin K Dichter
• 金额: $ 22.87万
• 依托单位: UNIVERSITY OF CALIF-LAWRENC BERKELEY LAB
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10827688
• 关键词: Address; Administrative Supplement; Adopted; Adoption; Archives; BRAIN initiative; Cloud Computing; Code; Communication; Communities; Computer Systems; Computer software; Consultations; Data; Data Analyses; Data Engineering; Data Set; Development; Education; Electrophysiology (science); Elements; Ensure; Environment; Evaluation; Expedite Dissemination; Funding Opportunities; Goals; Grant; Infrastructure; Laboratories; Libraries; Methods; Neurosciences; Output; Parents; Performance; Process; Publications; Publishing; Pythons; Readiness; Reading; Registries; Reproducibility; Research; Research Personnel; Resources; Running; Source; Standardization; System; Technology; Testing; Visualization; Work; Writing; base; cloud based; cloud storage; computerized data processing; cost; cost effective; data access; data exchange; data format; data management; data sharing; data standards; data tools; design; extracellular; hackathon; improved; innovative technologies; neurophysiology; new technology; parallel processing; parallelization; parent grant; parent project; rapid growth; response; software development; tool; usability; web services

PROJECT SUMMARY - Evaluation and optimization of NWB software and data in the cloud. This proposal is for an administrative supplement for the U24 grant “Advancing standardization of neurophysiology data through dissemination of NWB.” The parent project centers around providing support for the usage of Neurodata Without Borders (NWB), a data standard for neurophysiology data that allows neuroscience researchers to package and publish their data in a form that is readily available and reusable by others. Through the parent project, the team is taking several approaches to engage with the user community and lower the barriers to entry for adopting NWB, including hosted hackathons, one-on-one consultations, and tutorials. The team also ensures the continued quality of the NWB codebase through bug tracking, test coverage, and continued engagement with scientific software developers to assist with the integration of new tools for analysis, visualization, search, and publication of NWB datasets. Through our engagements with the community, we have identified the need to optimize NWB software and data for usage in the cloud as a key obstacle to adoption of NWB that we anticipate will become significant in the coming years. As neurophysiology data volumes continue to grow at a rapid pace, researchers are increasingly seeking to leverage the parallel processing capabilities of cloud infrastructure for converting data to NWB and analysis of NWB data. However, the current NWB conversion tools are not yet equipped for cloud integration and the NWB data layout is not optimized for cloud-based reading and analysis. To address these gaps, we will focus on two key aims. First, we will evaluate and optimize strategies for using cloud resources to enable researchers to perform efficient, cost-effective cloud-based conversion of data to NWB. Specifically, we will package our NeuroConv conversion software into containers that contain all of the necessary elements for NeuroConv to be run on any cloud computing environment, and we will develop tools for integrating existing cloud resources, e.g., for input and output of conversion data from/to cloud storage. Second, we will evaluate and optimize reading of NWB data from cloud storage to enhance cloud-based analysis. Specifically, we will integrate the Kerchunk software package designed to read data efficiently from the cloud with the PyNWB software for reading NWB data and we will evaluate the performance of different data layout strategies and optimize the storage of NWB data to enhance the efficiency for cloud-based access and analysis. Successful completion of the proposed work will create the necessary infrastructure and guidance for neuroscience researchers to take full advantage of cloud computing for conversion of data to NWB and analysis of data in NWB. This will enable researchers to convert and analyze their neurophysiology data faster and with fewer resources, which promises to improve data sharing and expedite scientific discovery.

项目摘要 - 云中 NWB 软件和数据的评估和优化。 本提案是对U24赠款“推进标准化”的行政补充 通过 NWB 传播神经生理学数据。”父项目的重点是为以下方面提供支持： 使用神经数据无国界 (NWB)，这是一种神经生理学数据的数据标准，允许 神经科学研究人员以易于获得和重复使用的形式打包和发布他们的数据 通过父项目，团队正在采取多种方法与用户社区互动。 降低采用 NWB 的准入门槛，包括举办黑客马拉松、一对一咨询以及 该团队还通过错误跟踪、测试来确保 NWB 代码库的持续质量。 覆盖范围，并继续与科学软件开发人员合作，以协助整合新的 用于分析、可视化、搜索和发布 NWB 数据集的工具。 通过与社区的合作，我们发现需要优化 NWB 软件和 在云中使用的数据是采用 NWB 的一个主要障碍，我们预计这将在以下领域变得重要： 随着神经生理学数据量继续快速增长，研究人员正在 越来越多地寻求利用云基础设施的并行处理能力来转换数据 到NWB并分析NWB数据但是，目前的NWB转换工具尚未配备云。 集成和 NWB 数据布局未针对基于云的读取和分析进行优化。 差距，我们将重点关注两个关键目标，首先，我们将评估和优化使用云资源的策略。 使研究人员能够执行高效、经济高效的基于云的数据到 NWB 的转换。 将我们的 NeuroConv 转换软件打包到包含所有必要元素的容器中 NeuroConv可以在任何云计算环境上运行，我们将开发用于集成现有的工具 云资源，例如，用于从云存储输入和输出转换数据。 其次，我们将评估。 优化从云存储中读取 NWB 数据，以增强基于云的分析。 将旨在从云端高效读取数据的 Kerchunk 软件包与 PyNWB 集成 用于读取 NWB 数据的软件，我们将评估不同数据布局策略的性能和 优化NWB数据存储，提高云端访问和分析效率。 成功完成拟议工作将为以下工作创造必要的基础设施和指导： 神经科学研究人员充分利用云计算将数据转换为 NWB 和 NWB 中的数据分析这将使研究人员能够更快地转换和分析他们的神经生理学数据。 并且资源更少，这有望改善数据共享并加速科学发现。