# Variational Causal Networks: Approximate Bayesian Inference over Causal Structures

arXiv: https://arxiv.org/abs/2106.07635 · HTML: https://mlsanity.com/paper/2106.07635/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [CAM: Causal additive models, high-dimensional order search and penalized regression](https://mlsanity.com/paper/1310.1533/) (2013)
- [A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms](https://mlsanity.com/paper/1901.10912/) (2019)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Learning Neural Causal Models from Unknown Interventions](https://mlsanity.com/paper/1910.01075/) (2019)
- [Backpropagation through the Void: Optimizing control variates for black-box gradient estimation](https://mlsanity.com/paper/1711.00123/) (2017)
- [Causal Structure Learning](https://mlsanity.com/paper/1706.09141/) (2017)
- [Integer Discrete Flows and Lossless Compression](https://mlsanity.com/paper/1905.07376/) (2019)
- [Discrete Flows: Invertible Generative Models of Discrete Data](https://mlsanity.com/paper/1905.10347/) (2019)
- [ABCD-Strategy: Budgeted Experimental Design for Targeted Causal Structure Discovery](https://mlsanity.com/paper/1902.10347/) (2019)
- [Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions](https://mlsanity.com/paper/1301.6697/) (2013)
- [Addendum on the scoring of Gaussian directed acyclic graphical models](https://mlsanity.com/paper/1402.6863/) (2014)
- [Partition MCMC for inference on acyclic digraphs](https://mlsanity.com/paper/1504.05006/) (2015)
- [Amortized learning of neural causal representations](https://mlsanity.com/paper/2008.09301/) (2020)
- [Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models](https://mlsanity.com/paper/1803.05554/) (2018)

## Built on by (most cited first, 30 of 43)

- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Bayesian Structure Learning with Generative Flow Networks](https://mlsanity.com/paper/2202.13903/) (2022)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [GFlowNets for AI-Driven Scientific Discovery](https://mlsanity.com/paper/2302.00615/) (2023)
- [Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network](https://mlsanity.com/paper/2305.19366/) (2023)
- [Interventions, Where and How? Experimental Design for Causal Models at Scale](https://mlsanity.com/paper/2203.02016/) (2022)
- [Learning Neural Causal Models with Active Interventions](https://mlsanity.com/paper/2109.02429/) (2021)
- [Active Bayesian Causal Inference](https://mlsanity.com/paper/2206.02063/) (2022)
- [BayesDAG: Gradient-Based Posterior Inference for Causal Discovery](https://mlsanity.com/paper/2307.13917/) (2023)
- [Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayes](https://mlsanity.com/paper/2211.02763/) (2022)
- [DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets](https://mlsanity.com/paper/2302.04178/) (2023)
- [BaCaDI: Bayesian Causal Discovery with Unknown Interventions](https://mlsanity.com/paper/2206.01665/) (2022)
- [A Meta-Learning Approach to Bayesian Causal Discovery](https://mlsanity.com/paper/2412.16577/) (2024)
- [GeneDisco: A Benchmark for Experimental Design in Drug Discovery](https://mlsanity.com/paper/2110.11875/) (2021)
- [Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning](https://mlsanity.com/paper/2406.14302/) (2024)
- [Tractable Uncertainty for Structure Learning](https://mlsanity.com/paper/2204.14170/) (2022)
- [Differentiable Multi-Target Causal Bayesian Experimental Design](https://mlsanity.com/paper/2302.10607/) (2023)
- [On the Generalization and Adaption Performance of Causal Models](https://mlsanity.com/paper/2206.04620/) (2022)
- [A Survey of Methods, Challenges and Perspectives in Causality](https://mlsanity.com/paper/2302.00293/) (2023)
- [Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders](https://mlsanity.com/paper/2402.14781/) (2024)
- [Learning Latent Structural Causal Models](https://mlsanity.com/paper/2210.13583/) (2022)
- [Trust Your nabla: Gradient-based Intervention Targeting for Causal Discovery](https://mlsanity.com/paper/2211.13715/) (2022)
- [Causal Inference in Gene Regulatory Networks with GFlowNet: Towards Scalability in Large Systems](https://mlsanity.com/paper/2310.03579/) (2023)
- [Challenges and Considerations in the Evaluation of Bayesian Causal Discovery](https://mlsanity.com/paper/2406.03209/) (2024)
- [CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design](https://mlsanity.com/paper/2302.14015/) (2023)
- [Permutation-based Inference for Variational Learning of Directed Acyclic Graphs](https://mlsanity.com/paper/2402.02644/) (2024)
- [ProDAG: Projected Variational Inference for Directed Acyclic Graphs](https://mlsanity.com/paper/2405.15167/) (2024)
- [Efficient Real-world Testing of Causal Decision Making via Bayesian Experimental Design for Contextual Optimisation](https://mlsanity.com/paper/2207.05250/) (2022)
- [Learning Multiscale Non-stationary Causal Structures](https://mlsanity.com/paper/2208.14989/) (2022)
- [Causal structure learning with momentum: Sampling distributions over Markov Equivalence Classes of DAGs](https://mlsanity.com/paper/2310.05655/) (2023)

## Abstract (arXiv)

Learning the causal structure that underlies data is a crucial step towards robust real-world decision making. The majority of existing work in causal inference focuses on determining a single directed acyclic graph (DAG) or a Markov equivalence class thereof. However, a crucial aspect to acting intelligently upon the knowledge about causal structure which has been inferred from finite data demands reasoning about its uncertainty. For instance, planning interventions to find out more about the causal mechanisms that govern our data requires quantifying epistemic uncertainty over DAGs. While Bayesian causal inference allows to do so, the posterior over DAGs becomes intractable even for a small number of variables. Aiming to overcome this issue, we propose a form of variational inference over the graphs of Structural Causal Models (SCMs). To this end, we introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs. Its number of parameters does not grow exponentially with the number of variables and can be tractably learned by maximising an Evidence Lower Bound (ELBO). In our experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.