CIF21 DIBBs: An Infrastructure Supporting Collaborative Data Analytics Workflow Design and Management

CIF21 DIBB：支持协作数据分析工作流程设计和管理的基础设施

• 批准号: 1443069
• 负责人: Jia Zhang
• 金额: $ 99.99万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-01-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1443069&HistoricalAwards=false
• 关键词: CIF21; DIBBs; Infrastructure; Supporting; Collaborative

The need for collaborative data analysis increases significantly when confronted with the challenges of big data. Although workflow tools offer a formal way to define, automate, and repeat multi-step computational procedures, designing complex data process workflow requires collaboration from multiple people with complementary expertise. Existing tools are not suitable to support collaborative design of comprehensive workflows. To address such a challenge, this project aims to design and develop a software infrastructure with the capability of supporting collaborative data-oriented workflow composition and management, adding a key component to existing NSF cyberinfrastructure that will support big data collaboration through the Internet. Reproducibility and scalability are two major targets. The project extends an existing open-source workflow tool, VisTrails, by adding system-level facilities to support human interaction and cooperation that are essential for an effective and efficient scientific collaboration.This project will produce five outcomes:1) A collaborative provenance data model equipped with a graph-level provenance querying formalism;2) A type-theoretic approach for addressing format transformations;3) Hypergraph theory-based algorithms for provenance management and mining;4) A software tool supporting (a)synchronous collaborative scientific workflow design, composition, reproduction, and visualization; and 5) Principles, methodologies, experiences, and lessons that support the development of a generically applicable collaborative scientific workflow composition tool.The resulting tools will explore the potential for using scientific workflows to accelerate scientific discoveries that require a collaborative effort on big data analytics. The design of the tools is targeted toward use cases in the civil engineering discipline, but has the potential to broadly impact other areas of science and engineering. Partnership with VisTrails enables usage and evaluation of the techniques in the VisTrails end user community.

当面对大数据的挑战时，对协作数据分析的需求大大增加。尽管Workflow工具提供了一种正式的方式来定义，自动化和重复多步计算程序，但设计复杂的数据过程工作流程需要多个具有互补专业知识的人的协作。 现有工具不适合支持综合工作流程的协作设计。 为了应对这一挑战，该项目旨在设计和开发软件基础架构，能够支持协作的面向数据的工作流程组成和管理，从而为现有的NSF Cyber​​infrststructure增加一个关键组成部分，该组件将支持通过Internet进行大型数据协作。 可重复性和可伸缩性是两个主要目标。 该项目通过添加系统级别的设施来支持人类的互动和合作，扩展了现有的开源工作流程工具，即Vistrail，这对于有效而有效的科学协作至关重要。该项目将产生五个结果：1）配备了与图形质量的形式的合作数据模型； 4）支持（a）同步协作科学工作流程设计，组成，复制和可视化的软件工具； 5）支持开发一般适用的协作科学工作流组成工具的原理，方法，经验和课程。所得工具将探索使用科学工作流程加速科学发现的潜力，以加速需要在大数据分析上进行协作工作的科学发现。 工具的设计针对土木工程纪律中的用例，但有可能广泛影响科学和工程的其他领域。 与Vistrail的合作伙伴关系可以使用和评估Vistrails最终用户社区中的技术。