mlsanityData to 7 Oct 2026

Challenges and Considerations in the Evaluation of Bayesian Causal Discovery

arXiv 2406.03209 ↗First posted 5 Jun 2024cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Representing uncertainty in causal discovery is a crucial component for experimental design, and more broadly, for safe and reliable causal decision making. Bayesian Causal Discovery (BCD) offers a principled approach to encapsulating this uncertainty. Unlike non-Bayesian causal discovery, which relies on a single estimated causal graph and model parameters for assessment, evaluating BCD presents challenges due to the nature of its inferred quantity - the posterior distribution. As a result, the research community has proposed various metrics to assess the quality of the approximate posterior. However, there is, to date, no consensus on the most suitable metric(s) for evaluation. In this work, we reexamine this question by dissecting various metrics and understanding their limitations. Through extensive empirical evaluation, we find that many existing metrics fail to exhibit a strong correlation with the quality of approximation to the true posterior, especially in scenarios with low sample sizes where BCD is most desirable. We highlight the suitability (or lack thereof) of these metrics under two distinct factors: the identifiability of the underlying causal model and the quantity of available data. Both factors affect the entropy of the true posterior, indicating that the current metrics are less fitting in settings of higher entropy. Our findings underline the importance of a more nuanced evaluation of new methods by taking into account the nature of the true posterior, as well as guide and motivate the development of new evaluation procedures for this challenge.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

28 of 53 references.

References · 28

  1. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  2. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  3. 2021Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Gamecited by 159
  4. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  5. 2019Variational Bayesian Optimal Experimental Designcited by 116
  6. 2021DiBS: Differentiable Bayesian Structure Learningcited by 105
  7. 2022Deep End-to-end Causal Inferencecited by 93
  8. 2022Amortized Inference for Causal Structure Learningcited by 89
  9. 2021BCD Nets: Scalable Variational Approaches for Bayesian Causal Discoverycited by 68
  10. 2019ABCD-Strategy: Budgeted Experimental Design for Targeted Causal Structure Discoverycited by 52
  11. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
  12. 2022Interventions, Where and How? Experimental Design for Causal Models at Scalecited by 49
16 more
  1. 2021Variational Causal Networks: Approximate Bayesian Inference over Causal Structurescited by 43
  2. 2013Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributionscited by 41
  3. 2022Active Bayesian Causal Inferencecited by 34
  4. 2023BayesDAG: Gradient-Based Posterior Inference for Causal Discoverycited by 33
  5. 2014Addendum on the scoring of Gaussian directed acyclic graphical modelscited by 32
  6. 2013Data Analysis with Bayesian Networks: A Bootstrap Approachcited by 31
  7. 2022Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayescited by 30
  8. 2023Assumption violations in causal discovery and the robustness of score matchingcited by 28
  9. 2023DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNetscited by 27
  10. 2022BaCaDI: Bayesian Causal Discovery with Unknown Interventionscited by 26
  11. 2022Model-based Causal Bayesian Optimizationcited by 26
  12. 2023Causal machine learning for single-cell genomicscited by 22
  13. 2013A Bayesian Approach to Learning Bayesian Networks with Local Structurecited by 15
  14. 2023Differentiable Multi-Target Causal Bayesian Experimental Designcited by 15
  15. 2023DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Designcited by 12
  16. 2023Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025CausalMan: A physics-based simulator for large-scale causalitycited by 4
  2. 2025Your Assumed DAG is Wrong and Here's How To Deal With Itcited by 1
  3. 2025A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inferencecited by 1
  4. 2025Cause-effect perception in an object place taskcited by 0
  5. 2026Causal Preference Elicitationcited by 0