# WfCommons: A Framework for Enabling Scientific Workflow Research and Development

arXiv: https://arxiv.org/abs/2105.14352 · HTML: https://mlsanity.com/paper/2105.14352/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2021.

## Built on by (most cited first, 8 of 8)

- [Towards Lightweight Data Integration using Multi-workflow Provenance and Data Observability](https://mlsanity.com/paper/2308.09004/) (2023)
- [MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems](https://mlsanity.com/paper/2112.07269/) (2021)
- [GCNScheduler: Scheduling Distributed Computing Applications using Graph Convolutional Networks](https://mlsanity.com/paper/2110.11552/) (2021)
- [Electricity Cost Minimization for Multi-Workflow Allocation in Geo-Distributed Data Centers](https://mlsanity.com/paper/2504.20105/) (2025)
- [LASER: Language Model Regression for Semi-Structured Workflow Resource and Runtime Estimation](https://mlsanity.com/paper/2512.19701/) (2025)
- [High-Performance Serverless Computing: A Systematic Literature Review on Serverless for HPC, AI, and Big Data](https://mlsanity.com/paper/2601.09334/) (2026)
- [LazyAgent: Demand-Driven Materialization and Physical Optimization of Agentic Programs](https://mlsanity.com/paper/2609.23058/) (2026)
- [Equal-Budget Monte Carlo Tree Search versus Randomized-Restart List Scheduling for DAG Workflows: A Controlled Equal-Budget Computational Study](https://mlsanity.com/paper/2610.12542/) (2026)

## Abstract (arXiv)

Scientific workflows are a cornerstone of modern scientific computing. They are used to describe complex computational applications that require efficient and robust management of large volumes of data, which are typically stored/processed on heterogeneous, distributed resources. The workflow research and development community has employed a number of methods for the quantitative evaluation of existing and novel workflow algorithms and systems. In particular, a common approach is to simulate workflow executions. In previous works, we have presented a collection of tools that have been adopted by the community for conducting workflow research. Despite their popularity, they suffer from several shortcomings that prevent easy adoption, maintenance, and consistency with the evolving structures and computational requirements of production workflows. In this work, we present WfCommons, a framework that provides a collection of tools for analyzing workflow executions, for producing generators of synthetic workflows, and for simulating workflow executions. We demonstrate the realism of the generated synthetic workflows by comparing their simulated executions to real workflow executions. We also contrast these results with results obtained when using the previously available collection of tools. We find that the workflow generators that are automatically constructed by our framework not only generate representative same-scale workflows (i.e., with structures and task characteristics distributions that resemble those observed in real-world workflows), but also do so at scales larger than that of available real-world workflows. Finally, we conduct a case study to demonstrate the usefulness of our framework for estimating the energy consumption of large-scale workflow executions.