Benchflow - A Benchmark for Workflow Management Systems

Benchflow - 工作流程管理系统的基准

• 批准号: 229974593
• 负责人: Professor Dr. Frank Leymann
• 金额: --
• 依托单位: Faculty of Informatics
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2013
• 资助国家: 德国
• 起止时间: 2012-12-31 至 2016-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/229974593?language=en
• 关键词: Benchflow; Benchmark; Workflow; Management; Systems

The goal of the BenchFlow project is to design the first benchmark for assessing and comparing the performance of workflow management systems (aka business process execution engines). Given the large number of heterogeneous systems and languages that have been proposed for modeling and executing workflows, to ensure the feasibility of this project, we will initially focus our efforts on a benchmark for standard WS-BPEL compliant Web service composition engines. Due to the inherent complexity of workflow engine architectures and the very large number of parameters affecting their performance, benchmarking workflow systems poses a number of scientific research challenges that require investigating a novel set of performance evaluation techniques. These will complement existing ones used to benchmark database management systems (e.g., TPC) or programming language compilers (e.g., SPEC), which are already well understood and have contributed to drive a very large performance improvement within the database, compiler and processor architecture industry. Workflow systems have become the platform to build composite service-oriented applications, whose performance depends on two factors: the performance of the workflow system itself and the performance of the composed services (which could lie outside of the control of the workflow). We plan to use a model-driven, self-/recursive testing approach to eliminate the impact of the external services by having them implemented as processes. The processes selected for the benchmark will be synthesized out of real-world WS-BPEL processes. Given the very large number of metrics that can be used to observe the performance of a workflow system, we aim at distilling a reduced set of performance indicators to compare different engines as well as different configurations of a given engine.

BenchFlow 项目的目标是设计第一个用于评估和比较工作流管理系统（又名业务流程执行引擎）性能的基准。鉴于已提出用于建模和执行工作流的大量异构系统和语言，为了确保该项目的可行性，我们首先将工作重点放在符合标准 WS-BPEL 的 Web 服务组合引擎的基准测试上。由于工作流引擎架构固有的复杂性以及影响其性能的大量参数，对工作流系统进行基准测试提出了许多科学研究挑战，需要研究一组新颖的性能评估技术。这些将补充用于对数据库管理系统（例如，TPC）或编程语言编译器（例如，SPEC）进行基准测试的现有基准，这些基准已经得到很好的理解，并有助于推动数据库、编译器和处理器架构行业的性能大幅提高。工作流系统已成为构建面向服务的组合应用程序的平台，其性能取决于两个因素：工作流系统本身的性能和组合服务的性能（可能超出工作流的控制范围）。我们计划使用模型驱动的自/递归测试方法，通过将外部服务实现为流程来消除外部服务的影响。 为基准测试选择的流程将从现实世界的 WS-BPEL 流程中综合出来。鉴于可用于观察工作流系统性能的指标数量非常多，我们的目标是提取一组精简的性能指标来比较不同的引擎以及给定引擎的不同配置。