# BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery

arXiv: https://arxiv.org/abs/2112.02761 · HTML: https://mlsanity.com/paper/2112.02761/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 73 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 22 of 22)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference](https://mlsanity.com/paper/1703.09194/) (2017)
- [DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks](https://mlsanity.com/paper/2010.09133/) (2020)
- [Learning linear structural equation models in polynomial time and sample complexity](https://mlsanity.com/paper/1707.04673/) (2017)
- [Towards Scalable Bayesian Learning of Causal DAGs](https://mlsanity.com/paper/2010.00684/) (2020)
- [Bayesian model averaging: A systematic review and conceptual classification](https://mlsanity.com/paper/1509.08864/) (2015)
- [Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope](https://mlsanity.com/paper/1904.05814/) (2019)
- [Locally Masked Convolution for Autoregressive Models](https://mlsanity.com/paper/2006.12486/) (2020)
- [On the Hyperprior Choice for the Global Shrinkage Parameter in the Horseshoe Prior](https://mlsanity.com/paper/1610.05559/) (2016)
- [Bayesian Network Structure Learning with Integer Programming: Polytopes, Facets, and Complexity](https://mlsanity.com/paper/1605.04071/) (2016)
- [Structure Learning of Linear Gaussian Structural Equation Models with Weak Edges](https://mlsanity.com/paper/1707.07560/) (2017)
- [A Tight Analysis of Bethe Approximation for Permanent](https://mlsanity.com/paper/1811.02933/) (2018)

## Built on by (most cited first, 30 of 68)

- [Bayesian Structure Learning with Generative Flow Networks](https://mlsanity.com/paper/2202.13903/) (2022)
- [A theory of continuous generative flow networks](https://mlsanity.com/paper/2301.12594/) (2023)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [GFlowNets for AI-Driven Scientific Discovery](https://mlsanity.com/paper/2302.00615/) (2023)
- [Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network](https://mlsanity.com/paper/2305.19366/) (2023)
- [Interventions, Where and How? Experimental Design for Causal Models at Scale](https://mlsanity.com/paper/2203.02016/) (2022)
- [CUTS+: High-dimensional Causal Discovery from Irregular Time-series](https://mlsanity.com/paper/2305.05890/) (2023)
- [Active Bayesian Causal Inference](https://mlsanity.com/paper/2206.02063/) (2022)
- [BayesDAG: Gradient-Based Posterior Inference for Causal Discovery](https://mlsanity.com/paper/2307.13917/) (2023)
- [Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes](https://mlsanity.com/paper/2211.02763/) (2022)
- [DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets](https://mlsanity.com/paper/2302.04178/) (2023)
- [BaCaDI: Bayesian Causal Discovery with Unknown Interventions](https://mlsanity.com/paper/2206.01665/) (2022)
- [A Meta-Learning Approach to Bayesian Causal Discovery](https://mlsanity.com/paper/2412.16577/) (2024)
- [Tractable Uncertainty for Structure Learning](https://mlsanity.com/paper/2204.14170/) (2022)
- [Differentiable Multi-Target Causal Bayesian Experimental Design](https://mlsanity.com/paper/2302.10607/) (2023)
- [Finding Counterfactually Optimal Action Sequences in Continuous State Spaces](https://mlsanity.com/paper/2306.03929/) (2023)
- [CoLiDE: Concomitant Linear DAG Estimation](https://mlsanity.com/paper/2310.02895/) (2023)
- [Structured Neural Networks for Density Estimation and Causal Inference](https://mlsanity.com/paper/2311.02221/) (2023)
- [Constraint-Free Structure Learning with Smooth Acyclic Orientations](https://mlsanity.com/paper/2309.08406/) (2023)
- [DAG Learning on the Permutahedron](https://mlsanity.com/paper/2301.11898/) (2023)
- [On the Generalization and Adaption Performance of Causal Models](https://mlsanity.com/paper/2206.04620/) (2022)
- [Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders](https://mlsanity.com/paper/2402.14781/) (2024)
- [Scalable Differentiable Causal Discovery in the Presence of Latent Confounders with Skeleton Posterior (Extended Version)](https://mlsanity.com/paper/2406.10537/) (2024)
- [Continuous Bayesian Model Selection for Multivariate Causal Discovery](https://mlsanity.com/paper/2411.10154/) (2024)
- [Learning Latent Structural Causal Models](https://mlsanity.com/paper/2210.13583/) (2022)
- [Trust Your nabla: Gradient-based Intervention Targeting for Causal Discovery](https://mlsanity.com/paper/2211.13715/) (2022)
- [Order-based Structure Learning with Normalizing Flows](https://mlsanity.com/paper/2308.07480/) (2023)
- [Causal Inference in Gene Regulatory Networks with GFlowNet: Towards Scalability in Large Systems](https://mlsanity.com/paper/2310.03579/) (2023)
- [Challenges and Considerations in the Evaluation of Bayesian Causal Discovery](https://mlsanity.com/paper/2406.03209/) (2024)
- [Structure Learning with Adaptive Random Neighborhood Informed MCMC](https://mlsanity.com/paper/2311.00599/) (2023)

## Abstract (arXiv)

A structural equation model (SEM) is an effective framework to reason over causal relationships represented via a directed acyclic graph (DAG). Recent advances have enabled effective maximum-likelihood point estimation of DAGs from observational data. However, a point estimate may not accurately capture the uncertainty in inferring the underlying graph in practical scenarios, wherein the true DAG is non-identifiable and/or the observed dataset is limited. We propose Bayesian Causal Discovery Nets (BCD Nets), a variational inference framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM. Developing a full Bayesian posterior over DAGs is challenging due to the the discrete and combinatorial nature of graphs. We analyse key design choices for scalable VI over DAGs, such as 1) the parametrization of DAGs via an expressive variational family, 2) a continuous relaxation that enables low-variance stochastic optimization, and 3) suitable priors over the latent variables. We provide a series of experiments on real and synthetic data showing that BCD Nets outperform maximum-likelihood methods on standard causal discovery metrics such as structural Hamming distance in low data regimes.