# Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning

arXiv: https://arxiv.org/abs/2507.05526 · HTML: https://mlsanity.com/paper/2507.05526/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 68 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning](https://mlsanity.com/paper/2106.02584/) (2021)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [DiBS: Differentiable Bayesian Structure Learning](https://mlsanity.com/paper/2105.11839/) (2021)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Amortized Inference for Causal Structure Learning](https://mlsanity.com/paper/2205.12934/) (2022)
- [Permutation-based Causal Inference Algorithms with Interventions](https://mlsanity.com/paper/1705.10220/) (2017)
- [Learning to Induce Causal Structure](https://mlsanity.com/paper/2204.04875/) (2022)
- [Understanding predictive information criteria for Bayesian models](https://mlsanity.com/paper/1307.5928/) (2013)
- [Differentiable DAG Sampling](https://mlsanity.com/paper/2203.08509/) (2022)
- [Causal Discovery with Score Matching on Additive Models with Arbitrary Noise](https://mlsanity.com/paper/2304.03265/) (2023)
- [Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions](https://mlsanity.com/paper/1301.6697/) (2013)
- [Active Bayesian Causal Inference](https://mlsanity.com/paper/2206.02063/) (2022)
- [Partition MCMC for inference on acyclic digraphs](https://mlsanity.com/paper/1504.05006/) (2015)
- [Towards Scalable Bayesian Learning of Causal DAGs](https://mlsanity.com/paper/2010.00684/) (2020)
- [Latent Bottlenecked Attentive Neural Processes](https://mlsanity.com/paper/2211.08458/) (2022)
- [A Meta-Learning Approach to Bayesian Causal Discovery](https://mlsanity.com/paper/2412.16577/) (2024)
- [A Bayesian Approach to Learning Causal Networks](https://mlsanity.com/paper/1302.4958/) (2013)
- [Standardizing Structural Causal Models](https://mlsanity.com/paper/2406.11601/) (2024)
- [Gaussian Process Networks](https://mlsanity.com/paper/1301.3857/) (2013)
- [Translation Equivariant Transformer Neural Processes](https://mlsanity.com/paper/2406.12409/) (2024)

## Built on by (most cited first, 13 of 13)

- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)
- [Use What You Know: Causal Foundation Models with Partial Graphs](https://mlsanity.com/paper/2602.14972/) (2026)
- [Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure](https://mlsanity.com/paper/2603.10254/) (2026)
- [Computational Identifiability](https://mlsanity.com/paper/2606.19361/) (2026)
- [PerturbPFN: Probing the Limits of Synthetic Priors in Drug Perturbation Modelling](https://mlsanity.com/paper/2607.23447/) (2026)
- [Foundation Models for Partial Causal Identification](https://mlsanity.com/paper/2608.20841/) (2026)
- [Causal Foundation Models](https://mlsanity.com/paper/2609.03003/) (2026)
- [PRIM: Meta-Learned Bayesian Root Cause Analysis](https://mlsanity.com/paper/2605.08786/) (2026)
- [Amortized Interventional Forecasting for Multivariate CIR Processes](https://mlsanity.com/paper/2608.03715/) (2026)
- [CAFE: Counterfactual Prediction via Fast Posterior Estimation](https://mlsanity.com/paper/2609.32167/) (2026)
- [CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimes](https://mlsanity.com/paper/2609.39523/) (2026)
- [Market Impact Estimation via Causal Foundation Models](https://mlsanity.com/paper/2610.12537/) (2026)
- [Amortized Counterfactual Prediction with Graph Neural Networks](https://mlsanity.com/paper/2610.13724/) (2026)

## Abstract (arXiv)

In scientific domains -- from biology to the social sciences -- many questions boil down to textitWhat effect will we observe if we intervene on a particular variable? If the causal relationships (e.g.~a causal graph) are known, it is possible to estimate the intervention distributions. In the absence of this domain knowledge, the causal structure must be discovered from the available observational data. However, observational data are often compatible with multiple causal graphs, making methods that commit to a single structure prone to overconfidence. A principled way to manage this structural uncertainty is via Bayesian inference, which averages over a posterior distribution on possible causal structures and functional mechanisms. Unfortunately, the number of causal structures grows super-exponentially with the number of nodes in the graph, making computations intractable. We propose to circumvent these challenges by using meta-learning to create an end-to-end model: the Model-Averaged Causal Estimation Transformer Neural Process (MACE-TNP). The model is trained to predict the Bayesian model-averaged interventional posterior distribution, and its end-to-end nature bypasses the need for expensive calculations. Empirically, we demonstrate that MACE-TNP outperforms strong Bayesian baselines. Our work establishes meta-learning as a flexible and scalable paradigm for approximating complex Bayesian causal inference, that can be scaled to increasingly challenging settings in the future.