# BayesDAG: Gradient-Based Posterior Inference for Causal Discovery

arXiv: https://arxiv.org/abs/2307.13917 · HTML: https://mlsanity.com/paper/2307.13917/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 78 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 30 of 43)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Bayesian Structure Learning with Generative Flow Networks](https://mlsanity.com/paper/2202.13903/) (2022)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [Foundations of Structural Causal Models with Cycles and Latent Variables](https://mlsanity.com/paper/1611.06221/) (2016)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks](https://mlsanity.com/paper/1512.07666/) (2015)
- [DiBS: Differentiable Bayesian Structure Learning](https://mlsanity.com/paper/2105.11839/) (2021)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Oops I Took A Gradient: Scalable Sampling for Discrete Distributions](https://mlsanity.com/paper/2102.04509/) (2021)
- [BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery](https://mlsanity.com/paper/2112.02761/) (2021)
- [SparseMAP: Differentiable Sparse Structured Inference](https://mlsanity.com/paper/1802.04223/) (2018)
- [DAGs with No Curl: An Efficient DAG Structure Learning Approach](https://mlsanity.com/paper/2106.07197/) (2021)
- [ABCD-Strategy: Budgeted Experimental Design for Targeted Causal Structure Discovery](https://mlsanity.com/paper/1902.10347/) (2019)
- [Interventions, Where and How? Experimental Design for Causal Models at Scale](https://mlsanity.com/paper/2203.02016/) (2022)
- [Differentiable DAG Sampling](https://mlsanity.com/paper/2203.08509/) (2022)
- [A Langevin-like Sampler for Discrete Distributions](https://mlsanity.com/paper/2206.09914/) (2022)
- [Variational Causal Networks: Approximate Bayesian Inference over Causal Structures](https://mlsanity.com/paper/2106.07635/) (2021)
- [Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions](https://mlsanity.com/paper/1301.6697/) (2013)
- [Variational Implicit Processes](https://mlsanity.com/paper/1806.02390/) (2018)
- [On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators](https://mlsanity.com/paper/1610.06665/) (2016)

## Built on by (most cited first, 30 of 33)

- [A Meta-Learning Approach to Bayesian Causal Discovery](https://mlsanity.com/paper/2412.16577/) (2024)
- [Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders](https://mlsanity.com/paper/2402.14781/) (2024)
- [Variational Search Distributions](https://mlsanity.com/paper/2409.06142/) (2024)
- [Continuous Bayesian Model Selection for Multivariate Causal Discovery](https://mlsanity.com/paper/2411.10154/) (2024)
- [Challenges and Considerations in the Evaluation of Bayesian Causal Discovery](https://mlsanity.com/paper/2406.03209/) (2024)
- [Permutation-based Inference for Variational Learning of Directed Acyclic Graphs](https://mlsanity.com/paper/2402.02644/) (2024)
- [ProDAG: Projected Variational Inference for Directed Acyclic Graphs](https://mlsanity.com/paper/2405.15167/) (2024)
- [Sample Efficient Bayesian Learning of Causal Graphs from Interventions](https://mlsanity.com/paper/2410.20089/) (2024)
- [Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery](https://mlsanity.com/paper/2502.16056/) (2025)
- [Differentiable Constraint-Based Causal Discovery](https://mlsanity.com/paper/2510.22031/) (2025)
- [Efficient Differentiable Discovery of Causal Order](https://mlsanity.com/paper/2410.08787/) (2024)
- [LOCAL: Learning with Orientation Matrix to Infer Causal Structure from Time Series Data](https://mlsanity.com/paper/2410.19464/) (2024)
- [Incorporating Expert Knowledge into Bayesian Causal Discovery of Mixtures of Directed Acyclic Graphs](https://mlsanity.com/paper/2510.06735/) (2025)
- [Test-Time Learning of Causal Structure from Interventional Data](https://mlsanity.com/paper/2602.19131/) (2026)
- [Bayesian Vector AutoRegression with Factorised Granger-Causal Graphs](https://mlsanity.com/paper/2402.03614/) (2024)
- [Scalable Variational Causal Discovery Unconstrained by Acyclicity](https://mlsanity.com/paper/2407.04992/) (2024)
- [Amortized Causal Discovery with Prior-Fitted Networks](https://mlsanity.com/paper/2512.11840/) (2025)
- [Consistency evaluation of benchmarks used for causal discovery](https://mlsanity.com/paper/2606.01789/) (2026)
- [Causal Bayesian Optimization with Unknown Graphs](https://mlsanity.com/paper/2503.19554/) (2025)
- [Large-Scale Bayesian Causal Discovery with Interventional Data](https://mlsanity.com/paper/2510.01562/) (2025)
- [Scaling Up Bayesian DAG Sampling](https://mlsanity.com/paper/2510.25254/) (2025)
- [Causal Preference Elicitation](https://mlsanity.com/paper/2602.01483/) (2026)
- [Moment Matters: Mean and Variance Causal Graph Discovery from Heteroscedastic Observational Data](https://mlsanity.com/paper/2602.23602/) (2026)
- [PRCD-MAP: Learning How Much to Trust Imperfect Priors in Causal Discovery](https://mlsanity.com/paper/2605.01669/) (2026)
- [A Topological Sorting Criterion for Random Causal Directed Acyclic Graphs](https://mlsanity.com/paper/2605.06288/) (2026)
- [A Differentiable Bayesian Relaxation for Latent Partial-Order Inference](https://mlsanity.com/paper/2605.06976/) (2026)
- [PACER: Acyclic Causal Discovery from Large-Scale Interventional Data](https://mlsanity.com/paper/2605.15353/) (2026)
- [Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk](https://mlsanity.com/paper/2605.29788/) (2026)
- [Learning Temporal Causal Structure via Smooth Differentiable Optimization](https://mlsanity.com/paper/2606.03227/) (2026)
- [Causal Atlases from Entropic Inference: Bayesian Networks beyond Optimal DAGs](https://mlsanity.com/paper/2606.06440/) (2026)

## Abstract (arXiv)

Bayesian causal discovery aims to infer the posterior distribution over causal models from observed data, quantifying epistemic uncertainty and benefiting downstream tasks. However, computational challenges arise due to joint inference over combinatorial space of Directed Acyclic Graphs (DAGs) and nonlinear functions. Despite recent progress towards efficient posterior inference over DAGs, existing methods are either limited to variational inference on node permutation matrices for linear causal models, leading to compromised inference accuracy, or continuous relaxation of adjacency matrices constrained by a DAG regularizer, which cannot ensure resulting graphs are DAGs. In this work, we introduce a scalable Bayesian causal discovery framework based on a combination of stochastic gradient Markov Chain Monte Carlo (SG-MCMC) and Variational Inference (VI) that overcomes these limitations. Our approach directly samples DAGs from the posterior without requiring any DAG regularization, simultaneously draws function parameter samples and is applicable to both linear and nonlinear causal models. To enable our approach, we derive a novel equivalence to the permutation-based DAG learning, which opens up possibilities of using any relaxed gradient estimator defined over permutations. To our knowledge, this is the first framework applying gradient-based MCMC sampling for causal discovery. Empirical evaluation on synthetic and real-world datasets demonstrate our approach's effectiveness compared to state-of-the-art baselines.