# Benchpress: A Scalable and Versatile Workflow for Benchmarking Structure Learning Algorithms

arXiv: https://arxiv.org/abs/2107.03863 · HTML: https://mlsanity.com/paper/2107.03863/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 107 references that are in the map.
- First posted on arXiv 8 Jul 2021.

## Builds on (most cited first, 18 of 18)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Learning Bayesian Networks: The Combination of Knowledge and Statistical Data](https://mlsanity.com/paper/1302.6815/) (2013)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [Permutation-based Causal Inference Algorithms with Interventions](https://mlsanity.com/paper/1705.10220/) (2017)
- [Learning Gaussian Networks](https://mlsanity.com/paper/1302.6808/) (2013)
- [Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions](https://mlsanity.com/paper/1301.6697/) (2013)
- [A Transformational Characterization of Equivalent Bayesian Network Structures](https://mlsanity.com/paper/1302.4938/) (2013)
- [Addendum on the scoring of Gaussian directed acyclic graphical models](https://mlsanity.com/paper/1402.6863/) (2014)
- [Partition MCMC for inference on acyclic digraphs](https://mlsanity.com/paper/1504.05006/) (2015)
- [Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data](https://mlsanity.com/paper/2005.09020/) (2020)
- [Efficient Sampling and Structure Learning of Bayesian Networks](https://mlsanity.com/paper/1803.07859/) (2018)
- [Bayesian structure learning and sampling of Bayesian networks with the R package BiDAG](https://mlsanity.com/paper/2105.00488/) (2021)
- [A Direct Sampler for G-Wishart Variates](https://mlsanity.com/paper/1304.1350/) (2013)
- [BDgraph: An R Package for Bayesian Structure Learning in Graphical Models](https://mlsanity.com/paper/1501.05108/) (2015)
- [algcomparison: Comparing the Performance of Graphical Structure Learning Algorithms with TETRAD](https://mlsanity.com/paper/1607.08110/) (2016)
- [A Birth and Death Process for Bayesian Network Structure Inference](https://mlsanity.com/paper/1610.00189/) (2016)
- [An Introduction to Rocker: Docker Containers for R](https://mlsanity.com/paper/1710.03675/) (2017)
- [On decomposable random graphs](https://mlsanity.com/paper/1710.03283/) (2017)

## Built on by (most cited first, 11 of 11)

- [A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models](https://mlsanity.com/paper/2303.18211/) (2023)
- [textttcausalAssembly: Generating Realistic Production Data for Benchmarking Causal Discovery](https://mlsanity.com/paper/2306.10816/) (2023)
- [Adjustment Identification Distance: A gadjid for Causal Structure Learning](https://mlsanity.com/paper/2402.08616/) (2024)
- [The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimation](https://mlsanity.com/paper/2303.01412/) (2023)
- [Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Method](https://mlsanity.com/paper/2405.13100/) (2024)
- [Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choice](https://mlsanity.com/paper/2310.18212/) (2023)
- [Embracing Discrete Search: A Reasonable Approach to Causal Structure Learning](https://mlsanity.com/paper/2510.04970/) (2025)
- [Causal Ordering for Structure Learning from Time Series](https://mlsanity.com/paper/2510.24639/) (2025)
- [Scalable Structure Learning for Sparse Context-Specific Systems](https://mlsanity.com/paper/2402.07762/) (2024)
- [Exact discovery is polynomial for certain sparse causal Bayesian networks](https://mlsanity.com/paper/2406.15012/) (2024)
- [Addressing pitfalls in implicit unobserved confounding synthesis using explicit block hierarchical ancestral sampling](https://mlsanity.com/paper/2503.09194/) (2025)

## Abstract (arXiv)

Describing the relationship between the variables in a study domain and modelling the data generating mechanism is a fundamental problem in many empirical sciences. Probabilistic graphical models are one common approach to tackle the problem. Learning the graphical structure for such models is computationally challenging and a fervent area of current research with a plethora of algorithms being developed. To facilitate the benchmarking of different methods, we present a novel Snakemake workflow, called Benchpress for producing scalable, reproducible, and platform-independent benchmarks of structure learning algorithms for probabilistic graphical models. Benchpress is interfaced via a simple JSON-file, which makes it accessible for all users, while the code is designed in a fully modular fashion to enable researchers to contribute additional methodologies. Benchpress currently provides an interface to a large number of state-of-the-art algorithms from libraries such as BDgraph, BiDAG, bnlearn, causal-learn, gCastle, GOBNILP, pcalg, r.blip, scikit-learn, TETRAD, and trilearn as well as a variety of methods for data generating models and performance evaluation. Alongside user-defined models and randomly generated datasets, the workflow also includes a number of standard datasets and graphical models from the literature, which may be included in a benchmarking study. We demonstrate the applicability of this workflow for learning Bayesian networks in five typical data scenarios. The source code and documentation is publicly available from http://benchpressdocs.readthedocs.io.