# Are you doing better than random guessing? A call for using negative controls when evaluating causal discovery algorithms

arXiv: https://arxiv.org/abs/2412.10039 · HTML: https://mlsanity.com/paper/2412.10039/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Greedy Relaxations of the Sparsest Permutation Algorithm](https://mlsanity.com/paper/2206.05421/) (2022)
- [Fast Scalable and Accurate Discovery of DAGs Using the Best Order Score Search and Grow-Shrink Trees](https://mlsanity.com/paper/2310.17679/) (2023)
- [The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data](https://mlsanity.com/paper/1910.05387/) (2019)
- [Supervised Whole DAG Causal Discovery](https://mlsanity.com/paper/2006.04697/) (2020)
- [Adjustment Identification Distance: A gadjid for Causal Structure Learning](https://mlsanity.com/paper/2402.08616/) (2024)
- [Toward Falsifying Causal Graphs Using a Permutation-Based Test](https://mlsanity.com/paper/2305.09565/) (2023)
- [Causal discovery for observational sciences using supervised machine learning](https://mlsanity.com/paper/2202.12813/) (2022)
- [Separation-based distance measures for causal graphs](https://mlsanity.com/paper/2402.04952/) (2024)
- [Improving Finite Sample Performance of Causal Discovery by Exploiting Temporal Structure](https://mlsanity.com/paper/2406.19503/) (2024)

## Built on by (most cited first, 5 of 5)

- [Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption](https://mlsanity.com/paper/2508.08883/) (2025)
- [How PC-based Methods Err: Towards Better Reporting of Assumption Violations and Small Sample Errors](https://mlsanity.com/paper/2502.14719/) (2025)
- [Robust Causal Discovery in Real-World Time Series with Power-Laws](https://mlsanity.com/paper/2507.12257/) (2025)
- [AutoCause: A Python framework that automates expert decisions in environmental time-series causal discovery](https://mlsanity.com/paper/2608.00198/) (2026)
- [Beyond Stationarity in Time Series: Discovering Causal Structures and Latent Regimes via Markov Blankets](https://mlsanity.com/paper/2609.05150/) (2026)

## Abstract (arXiv)

New proposals for causal discovery algorithms are typically evaluated using simulations and a few selected real data examples with known data generating mechanisms. However, there does not exist a general guideline for how such evaluation studies should be designed, and therefore, comparing results across different studies can be difficult. In this article, we propose to use negative controls as a common evaluation baseline by posing the question: Are we doing better than random guessing? For the task of graph skeleton estimation, we derive exact distributional results under random guessing for the expected behavior of a range of typical causal discovery evaluation metrics, including precision and recall. We show that these metrics can achieve very favorable values under random guessing in certain scenarios, and hence warn against using them without also reporting negative control results, i.e., performance under random guessing. We also propose an exact test of overall skeleton fit, and showcase its use on a real data application. Finally, we propose a general pipeline for using negative controls beyond the skeleton estimation task, and apply it both in a simulated example and a real data application.