Evaluation and optimization of NWB neurophysiology software and data in the cloud

NWB 神经生理学软件和云数据的评估和优化

• 批准号: 10827688
• 负责人: Benjamin K Dichter
• 金额: $ 22.87万
• 依托单位: UNIVERSITY OF CALIF-LAWRENC BERKELEY LAB
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10827688
• 关键词: Address; Administrative Supplement; Adopted; Adoption; Archives; BRAIN initiative; Cloud Computing; Code; Communication; Communities; Computer Systems; Computer software; Consultations; Data; Data Analyses; Data Engineering; Data Set; Development; Education; Electrophysiology (science); Elements; Ensure; Environment; Evaluation; Expedite Dissemination; Funding Opportunities; Goals; Grant; Infrastructure; Laboratories; Libraries; Methods; Neurosciences; Output; Parents; Performance; Process; Publications; Publishing; Pythons; Readiness; Reading; Registries; Reproducibility; Research; Research Personnel; Resources; Running; Source; Standardization; System; Technology; Testing; Visualization; Work; Writing; base; cloud based; cloud storage; computerized data processing; cost; cost effective; data access; data exchange; data format; data management; data sharing; data standards; data tools; design; extracellular; hackathon; improved; innovative technologies; neurophysiology; new technology; parallel processing; parallelization; parent grant; parent project; rapid growth; response; software development; tool; usability; web services

PROJECT SUMMARY - Evaluation and optimization of NWB software and data in the cloud. This proposal is for an administrative supplement for the U24 grant “Advancing standardization of neurophysiology data through dissemination of NWB.” The parent project centers around providing support for the usage of Neurodata Without Borders (NWB), a data standard for neurophysiology data that allows neuroscience researchers to package and publish their data in a form that is readily available and reusable by others. Through the parent project, the team is taking several approaches to engage with the user community and lower the barriers to entry for adopting NWB, including hosted hackathons, one-on-one consultations, and tutorials. The team also ensures the continued quality of the NWB codebase through bug tracking, test coverage, and continued engagement with scientific software developers to assist with the integration of new tools for analysis, visualization, search, and publication of NWB datasets. Through our engagements with the community, we have identified the need to optimize NWB software and data for usage in the cloud as a key obstacle to adoption of NWB that we anticipate will become significant in the coming years. As neurophysiology data volumes continue to grow at a rapid pace, researchers are increasingly seeking to leverage the parallel processing capabilities of cloud infrastructure for converting data to NWB and analysis of NWB data. However, the current NWB conversion tools are not yet equipped for cloud integration and the NWB data layout is not optimized for cloud-based reading and analysis. To address these gaps, we will focus on two key aims. First, we will evaluate and optimize strategies for using cloud resources to enable researchers to perform efficient, cost-effective cloud-based conversion of data to NWB. Specifically, we will package our NeuroConv conversion software into containers that contain all of the necessary elements for NeuroConv to be run on any cloud computing environment, and we will develop tools for integrating existing cloud resources, e.g., for input and output of conversion data from/to cloud storage. Second, we will evaluate and optimize reading of NWB data from cloud storage to enhance cloud-based analysis. Specifically, we will integrate the Kerchunk software package designed to read data efficiently from the cloud with the PyNWB software for reading NWB data and we will evaluate the performance of different data layout strategies and optimize the storage of NWB data to enhance the efficiency for cloud-based access and analysis. Successful completion of the proposed work will create the necessary infrastructure and guidance for neuroscience researchers to take full advantage of cloud computing for conversion of data to NWB and analysis of data in NWB. This will enable researchers to convert and analyze their neurophysiology data faster and with fewer resources, which promises to improve data sharing and expedite scientific discovery.

项目摘要 - 云中NWB软件和数据的评估和优化。 该提案是为U24赠款的行政补充 神经生理学数据是通过传播NWB的。 无边界的神经逻辑（NWB）的使用，NWB是神经生理学数据的数据标准 神经科学的研究人员以容易获得并可以重复使用的形式包装和发布其数据 其他的。通过父项目，团队正在采取几种方法与用户社区互动 并降低采用NWB的进入障碍，包括托管黑客马拉松，一对一的咨询和 教程。该团队还通过错误跟踪确保NWB代码库的持续质量 覆盖范围，并继续与科学软件开发人员互动，以协助整合新的 用于分析，可视化，搜索和发布NWB数据集的工具。 通过与社区的交往，我们确定了优化NWB软件和 在云中使用的数据是我们预期采用NWB的关键障碍 未来几年。随着神经生理学的数据量继续以快速的速度增长，研究人员就是 越来越多地寻求利用云基础架构的并行处理能力来转换数据 进行NWB和NWB数据的分析。但是，当前的NWB转换工具尚未配备云 集成和NWB数据布局未针对基于云的阅读和分析进行优化。解决这些 空白，我们将重点关注两个关键目标。首先，我们将评估并优化使用云资源的策略 使研究人员能够对数据进行有效的，具有成本效益的基于云的转换为NWB。具体来说，我们 将我们的NeuroConv转换软件包装到包含所有必要元素的容器中 NeuroConv将在任何云计算环境上运行，我们将开发用于集成现有的工具 云资源，例如，用于从/到云存储的转换数据的输入和输出。第二，我们将评估 并优化从云存储中读取NWB数据以增强基于云的分析。具体来说，我们会的 集成了Kerchunk软件包，旨在用Pynwb从云中有效地读取数据 用于读取NWB数据的软件，我们将评估不同数据布局策略的性能和 优化NWB数据的存储，以提高基于云的访问和分析的效率。 成功完成拟议的工作将创建必要的基础设施和指导 神经科学研究人员将充分利用云计算，以将数据转换为NWB和 NWB中的数据分析。这将使研究人员能够更快地转换和分析其神经生理学数据 并且资源较少，这有望改善数据共享并加快科学发现。