# Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Method

arXiv: https://arxiv.org/abs/2405.13100 · HTML: https://mlsanity.com/paper/2405.13100/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 70 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 22 of 22)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization](https://mlsanity.com/paper/2209.08037/) (2022)
- [Causal Networks: Semantics and Expressiveness](https://mlsanity.com/paper/1304.2379/) (2013)
- [Greedy Relaxations of the Sparsest Permutation Algorithm](https://mlsanity.com/paper/2206.05421/) (2022)
- [Unsuitability of NOTEARS for Causal Graph Discovery](https://mlsanity.com/paper/2104.05441/) (2021)
- [Strong Completeness and Faithfulness in Bayesian Networks](https://mlsanity.com/paper/1302.4973/) (2013)
- [Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions](https://mlsanity.com/paper/1301.6697/) (2013)
- [Fast Scalable and Accurate Discovery of DAGs Using the Best Order Score Search and Grow-Shrink Trees](https://mlsanity.com/paper/2310.17679/) (2023)
- [Structure Learning with Continuous Optimization: A Sober Look and Beyond](https://mlsanity.com/paper/2304.02146/) (2023)
- [Addendum on the scoring of Gaussian directed acyclic graphical models](https://mlsanity.com/paper/1402.6863/) (2014)
- [Neuropathic Pain Diagnosis Simulator for Causal Discovery Algorithm Evaluation](https://mlsanity.com/paper/1906.01732/) (2019)
- [Efficient Sampling and Structure Learning of Bayesian Networks](https://mlsanity.com/paper/1803.07859/) (2018)
- [A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models](https://mlsanity.com/paper/2303.18211/) (2023)
- [Assumption violations in causal discovery and the robustness of score matching](https://mlsanity.com/paper/2310.13387/) (2023)
- [Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data](https://mlsanity.com/paper/2104.08043/) (2021)
- [Bayesian structure learning and sampling of Bayesian networks with the R package BiDAG](https://mlsanity.com/paper/2105.00488/) (2021)
- [Benchpress: A Scalable and Versatile Workflow for Benchmarking Structure Learning Algorithms](https://mlsanity.com/paper/2107.03863/) (2021)
- [A Uniformly Consistent Estimator of Causal Effects under the k-Triangle-Faithfulness Assumption](https://mlsanity.com/paper/1502.00829/) (2015)
- [algcomparison: Comparing the Performance of Graphical Structure Learning Algorithms with TETRAD](https://mlsanity.com/paper/1607.08110/) (2016)
- [A Comparison of Public Causal Search Packages on Linear, Gaussian Data with No Latent Variables](https://mlsanity.com/paper/1709.04240/) (2017)
- [A Uniformly Consistent Estimator of non-Gaussian Causal Effects Under the k-Triangle-Faithfulness Assumption](https://mlsanity.com/paper/2107.01333/) (2021)

## Built on by (most cited first, 6 of 6)

- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [Unitless Unrestricted Markov-Consistent SCM Generation: Better Benchmark Datasets for Causal Discovery](https://mlsanity.com/paper/2503.17037/) (2025)
- [Embracing Discrete Search: A Reasonable Approach to Causal Structure Learning](https://mlsanity.com/paper/2510.04970/) (2025)
- [Choosing DAG Models Using Markov and Minimal Edge Count in the Absence of Ground Truth](https://mlsanity.com/paper/2409.20187/) (2024)
- [An extensive simulation study evaluating the interaction of resampling techniques across multiple causal discovery contexts](https://mlsanity.com/paper/2503.15436/) (2025)
- [A Topological Sorting Criterion for Random Causal Directed Acyclic Graphs](https://mlsanity.com/paper/2605.06288/) (2026)

## Abstract (arXiv)

The number of artificial intelligence algorithms for learning causal models from data is growing rapidly. Most ``causal discovery'' or ``causal structure learning'' algorithms are primarily validated through simulation studies. However, no widely accepted simulation standards exist and publications often report conflicting performance statistics -- even when only considering publications that simulate data from linear models. In response, several manuscripts have criticized a popular simulation design for validating algorithms in the linear case. We propose a new simulation design for generating linear models for directed acyclic graphs (DAGs): the DAG-adaptation of the Onion (DaO) method. DaO simulations are fundamentally different from existing simulations because they prioritize the distribution of correlation matrices rather than the distribution of linear effects. Specifically, the DaO method uniformly samples the space of all correlation matrices consistent with (i.e. Markov to) a DAG. We also discuss how to sample DAGs and present methods for generating DAGs with scale-free in-degree or out-degree. We compare the DaO method against two alternative simulation designs and provide implementations of the DaO method in Python and R: https://github.com/bja43/DaO_simulation. We advocate for others to adopt DaO simulations as a fair universal benchmark.