# An extensive simulation study evaluating the interaction of resampling techniques across multiple causal discovery contexts

arXiv: https://arxiv.org/abs/2503.15436 · HTML: https://mlsanity.com/paper/2503.15436/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 7 of 7)

- [Fast Scalable and Accurate Discovery of DAGs Using the Best Order Score Search and Grow-Shrink Trees](https://mlsanity.com/paper/2310.17679/) (2023)
- [Bootstrap aggregation and confidence measures to improve time series causal discovery](https://mlsanity.com/paper/2306.08946/) (2023)
- [Py-Tetrad and RPy-Tetrad: A New Python Interface with R Support for Tetrad Causal Search](https://mlsanity.com/paper/2308.07346/) (2023)
- [Confidence Sets for Causal Orderings](https://mlsanity.com/paper/2305.14506/) (2023)
- [Better Simulations for Validating Causal Discovery with the DAG-Adaptation of the Onion Method](https://mlsanity.com/paper/2405.13100/) (2024)
- [Simulations evaluating resampling methods for causal discovery: ensemble performance and calibration](https://mlsanity.com/paper/1910.02047/) (2019)
- [Obtaining Accurate Probabilistic Causal Inference by Post-Processing Calibration](https://mlsanity.com/paper/1712.08626/) (2017)

## Abstract (arXiv)

Despite the accelerating presence of exploratory causal analysis in modern science and medicine, the available non-experimental methods for validating causal models are not well characterized. One of the most popular methods is to evaluate the stability of model features after resampling the data, similar to resampling methods for estimating confidence intervals in statistics. Many aspects of this approach have received little to no attention, however, such as whether the choice of resampling method should depend on the sample size, algorithms being used, or algorithm tuning parameters. We present theoretical results proving that certain resampling methods closely emulate the assignment of specific values to algorithm tuning parameters. We also report the results of extensive simulation experiments, which verify the theoretical result and provide substantial data to aid researchers in further characterizing resampling in the context of causal discovery analysis. Together, the theoretical work and simulation results provide specific guidance on how resampling methods and tuning parameters should be selected in practice.