# Toward Falsifying Causal Graphs Using a Permutation-Based Test

arXiv: https://arxiv.org/abs/2305.09565 · HTML: https://mlsanity.com/paper/2305.09565/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 62 references that are in the map.
- First posted on arXiv 16 May 2023.

## Builds on (most cited first, 14 of 14)

- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [CAM: Causal additive models, high-dimensional order search and penalized regression](https://mlsanity.com/paper/1310.1533/) (2013)
- [Bayesian Structure Learning with Generative Flow Networks](https://mlsanity.com/paper/2202.13903/) (2022)
- [Score matching enables causal discovery of nonlinear additive noise models](https://mlsanity.com/paper/2203.04413/) (2022)
- [DiBS: Differentiable Bayesian Structure Learning](https://mlsanity.com/paper/2105.11839/) (2021)
- [Causal structure based root cause analysis of outliers](https://mlsanity.com/paper/1912.02724/) (2019)
- [Visual Causal Feature Learning](https://mlsanity.com/paper/1412.2309/) (2014)
- [Learning Sparse Causal Models is not NP-hard](https://mlsanity.com/paper/1309.6824/) (2013)
- [Strong Completeness and Faithfulness in Bayesian Networks](https://mlsanity.com/paper/1302.4973/) (2013)
- [DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models](https://mlsanity.com/paper/2206.06821/) (2022)
- [FASK with Interventional Knowledge Recovers Edges from the Sachs Model](https://mlsanity.com/paper/1805.03108/) (2018)
- [Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentation](https://mlsanity.com/paper/2103.00136/) (2021)
- [P-values: misunderstood and misused](https://mlsanity.com/paper/1601.06805/) (2016)

## Built on by (most cited first, 11 of 11)

- [Self-Compatibility: Evaluating Causal Discovery without Ground Truth](https://mlsanity.com/paper/2307.09552/) (2023)
- [Root Cause Analysis of Outliers with Missing Structural Knowledge](https://mlsanity.com/paper/2406.05014/) (2024)
- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [The PetShop Dataset -- Finding Causes of Performance Issues across Microservices](https://mlsanity.com/paper/2311.04806/) (2023)
- [Separation-based distance measures for causal graphs](https://mlsanity.com/paper/2402.04952/) (2024)
- [Are you doing better than random guessing? A call for using negative controls when evaluating causal discovery algorithms](https://mlsanity.com/paper/2412.10039/) (2024)
- [On Different Notions of Redundancy in Conditional-Independence-Based Discovery of Graphical Models](https://mlsanity.com/paper/2502.08531/) (2025)
- [Consistency evaluation of benchmarks used for causal discovery](https://mlsanity.com/paper/2606.01789/) (2026)
- [Evaluating Bivariate Causal Statements Based on Mutual Compatibility](https://mlsanity.com/paper/2606.00278/) (2026)
- [A Step Towards Inherently Interpretable Causal Machine Learning Models For Decision Support](https://mlsanity.com/paper/2606.24348/) (2026)
- [Falsifying Causal Graphs With Outlier Events](https://mlsanity.com/paper/2607.12145/) (2026)

## Abstract (arXiv)

Understanding causal relationships among the variables of a system is paramount to explain and control its behavior. For many real-world systems, however, the true causal graph is not readily available and one must resort to predictions made by algorithms or domain experts. Therefore, metrics that quantitatively assess the goodness of a causal graph provide helpful checks before using it in downstream tasks. Existing metrics provide an textitabsolute number of inconsistencies between the graph and the observed data, and without a baseline, practitioners are left to answer the hard question of how many such inconsistencies are acceptable or expected. Here, we propose a novel consistency metric by constructing a baseline through node permutations. By comparing the number of inconsistencies with those on the baseline, we derive an interpretable metric that captures whether the graph is significantly better than random. Evaluating on both simulated and real data sets from various domains, including biology and cloud monitoring, we demonstrate that the true graph is not falsified by our metric, whereas the wrong graphs given by a hypothetical user are likely to be falsified.