mlsanityData to 7 Oct 2026

Benchpress: A Scalable and Versatile Workflow for Benchmarking Structure Learning Algorithms

arXiv 2107.03863 ↗First posted 8 Jul 2021stat.ML cs.LG stat.APcited by 11

Not on the map. Why?

Abstract (arXiv)

Describing the relationship between the variables in a study domain and modelling the data generating mechanism is a fundamental problem in many empirical sciences. Probabilistic graphical models are one common approach to tackle the problem. Learning the graphical structure for such models is computationally challenging and a fervent area of current research with a plethora of algorithms being developed. To facilitate the benchmarking of different methods, we present a novel Snakemake workflow, called Benchpress for producing scalable, reproducible, and platform-independent benchmarks of structure learning algorithms for probabilistic graphical models. Benchpress is interfaced via a simple JSON-file, which makes it accessible for all users, while the code is designed in a fully modular fashion to enable researchers to contribute additional methodologies. Benchpress currently provides an interface to a large number of state-of-the-art algorithms from libraries such as BDgraph, BiDAG, bnlearn, causal-learn, gCastle, GOBNILP, pcalg, r.blip, scikit-learn, TETRAD, and trilearn as well as a variety of methods for data generating models and performance evaluation. Alongside user-defined models and randomly generated datasets, the workflow also includes a number of standard datasets and graphical models from the literature, which may be included in a benchmarking study. We demonstrate the applicability of this workflow for learning Bayesian networks in five typical data scenarios. The source code and documentation is publicly available from http://benchpressdocs.readthedocs.io.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 18 of its 107 references that are in the map. First posted on arXiv 8 Jul 2021.

Builds on

18 of 107 references.

References · 18

  1. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
  2. 2013Learning Bayesian Networks: The Combination of Knowledge and Statistical Datacited by 185
  3. 2021Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Gamecited by 159
  4. 2017Permutation-based Causal Inference Algorithms with Interventionscited by 78
  5. 2013Learning Gaussian Networkscited by 62
  6. 2013Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributionscited by 41
  7. 2013A Transformational Characterization of Equivalent Bayesian Network Structurescited by 41
  8. 2015Partition MCMC for inference on acyclic digraphscited by 33
  9. 2014Addendum on the scoring of Gaussian directed acyclic graphical modelscited by 32
  10. 2020Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy datacited by 29
  11. 2018Efficient Sampling and Structure Learning of Bayesian Networkscited by 28
  12. 2021Bayesian structure learning and sampling of Bayesian networks with the R package BiDAGcited by 17
6 more
  1. 2013A Direct Sampler for G-Wishart Variatescited by 4
  2. 2015BDgraph: An R Package for Bayesian Structure Learning in Graphical Modelscited by 4
  3. 2016algcomparison: Comparing the Performance of Graphical Structure Learning Algorithms with TETRADcited by 3
  4. 2016A Birth and Death Process for Bayesian Network Structure Inferencecited by 2
  5. 2017An Introduction to Rocker: Docker Containers for Rcited by 2
  6. 2017On decomposable random graphscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 4; 2025: 3.

11 papers.

  1. 2023A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Modelscited by 28
  2. 2023textttcausalAssembly: Generating Realistic Production Data for Benchmarking Causal Discoverycited by 18
  3. 2024Adjustment Identification Distance: A gadjid for Causal Structure Learningcited by 16
  4. 2023The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimationcited by 6
  5. 2024Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Methodcited by 6
  6. 2023Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choicecited by 4
  7. 2025Embracing Discrete Search: A Reasonable Approach to Causal Structure Learningcited by 4
  8. 2025Causal Ordering for Structure Learning from Time Seriescited by 1
  9. 2024Scalable Structure Learning for Sparse Context-Specific Systemscited by 0
  10. 2024Exact discovery is polynomial for certain sparse causal Bayesian networkscited by 0
  11. 2025Addressing pitfalls in implicit unobserved confounding synthesis using explicit block hierarchical ancestral samplingcited by 0