A modular data analysis ecosystem using portable encapsulated projects

使用便携式封装项目的模块化数据分析生态系统

• 批准号: 9751344
• 负责人: Nathan Sheffield
• 金额: $ 39.32万
• 依托单位: UNIVERSITY OF VIRGINIA
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9751344
• 关键词: Adopted; Bioinformatics; Biomedical Research; Complex; Data; Data Analyses; Data Collection; Data Set; Ecosystem; Encapsulated; Environment; Goals; Human; Individual; Inherited; Institution; Knowledge; Link; Manuals; Process; Provider; Pythons; Research; Research Project Grants; Running; Series; Standardization; Structure; Surface; System; Techniques; base; bioinformatics tool; cluster computing; computerized data processing; data management; data sharing; innovation; insight; interest; next generation; novel; novel strategies; portability; tool; tool development

Project summary Overview As the amount of available data increases, it becomes more challenging to process it. Data processing is simple on the surface: it is a mapping from data to analysis. Unfortunately, too often, this requires a unique structure for each combination of dataset and analysis. This makes it difficult to do things like run several different analyses on one dataset, or plug several different datasets to one analysis, because each connection structure must be defined manually. To alleviate this challenge of linking data to tools, this proposal develops the concept of Portable Encapsulated Projects (PEP) and a series of tools that read and process such projects. Essentially, the PEP format aims to standardize the description of data collections, enabling both data providers and data users to communicate through the common interface of a standard format. Practically, this means individuals who describe their projects using this format will immediately inherit both greater portability for analysis as well as greater access to external complementary data. This link operates around a simple, standard, extensible definition of a project. Accompanying this, this proposal develops Python and R packages to provide a modular framework with a low barrier to entry that makes it easy to build robust pipelines and other tools centered around the PEP format. This system presents a new approach to organizing data-intensive biomedical research projects. Significance and innovation This proposal sits at the interface of data management and bioinformatics tool development. While significant effort is already dedicated to each of these individually, there has been less focus at the level of connecting the two. This proposal will build a standardized interface between data and tools in bioinformatics, providing practical advances in formats and tools to facilitate this interaction. This effort approaches computational projects in a novel way, and builds both concepts and tools that can revolutionize bioinformatics research. The goal is not to develop new tools, but to make existing tools more easily applied to existing data. In computational research, a huge amount of effort is spent in data cleanup: preparing data for analysis. By facilitating the connection from data to tools, this will encourage re-analysis of existing data with novel analysis techniques, leading to new discovery. It will also make it easier to analyze new data in tandem with existing data, increasing the value of both. It will contribute to reusability, larger-scale analysis, portable computing environments, and data sharing. There is increasing interest in data sharing and accessibility across scientific domains, and this proposal will facilitate this. Early versions are already adopted for both local compute and cluster computing at four different research institutions, and as the project matures, it will unite various research environments around a common data description. This will make it easier to share data and tools across users, research groups, and institutions. 1

项目概要 概述 随着可用数据量的增加，处理数据变得更加困难。数据处理很简单。 表面：它是从数据到分析的映射，不幸的是，这通常需要每个组合都有一个独特的结构。 这使得在一个数据集上运行多个不同的分析或插入多个数据集变得困难。 不同的数据集进行一次分析，因为必须手动定义每个连接结构。 为了缓解将数据链接到工具的这一挑战，该提案开发了可移植封装项目的概念 (PEP) 以及一系列读取和处理此类项目的工具 本质上，PEP 格式旨在标准化项目。 数据收集的描述，使数据提供者和数据使用者能够通过通用接口进行通信 实际上，这意味着使用这种格式描述其项目的个人将立即继承。 不仅具有更大的分析可移植性，而且还能更好地访问外部补充数据。 项目的简单、标准、可扩展的定义。 与此同时，该提案开发了 Python 和 R 包，以提供一个低门槛的模块化框架。 可以轻松构建以 PEP 格式为中心的强大管道和其他工具。 组织数据密集型生物医学研究项目的新方法。 意义与创新 该提案位于数据管理和生物信息学工具开发的界面，同时付出了巨大的努力。 已经分别致力于其中的每一个，因此在连接两者的层面上关注较少。 将在生物信息学数据和工具之间建立标准化接口，提供格式和工具方面的实用进步 为了促进这种交互，这项工作以一种新颖的方式处理计算项目，并构建了概念和工具。 能够彻底改变生物信息学研究的目标不是开发新工具，而是使现有工具更容易使用。 应用于现有数据。 在计算研究中，大量的精力花费在数据清理上：通过促进数据准备来进行分析。 从数据到工具的连接，这将鼓励使用新颖的分析技术重新分析现有数据，从而产生新的 它还将使新数据与现有数据的分析变得更加容易，从而增加两者的价值。 有助于可重用性、大规模分析、便携式计算环境和数据共享。 人们对跨科学领域的数据共享和可访问性越来越感兴趣，该提案将促进这一点。 早期版本已被四个不同的研究机构采用用于本地计算和集群计算，并且 随着项目的成熟，它将围绕一个共同的数据描述将各种研究环境联合起来。 更容易在用户、研究小组和机构之间共享数据和工具。 1