mlsanityData to 14 Oct 2026

Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning

arXiv 2507.05526 ↗First posted 7 Jul 2025cs.LG stat.ME stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

In scientific domains -- from biology to the social sciences -- many questions boil down to textitWhat effect will we observe if we intervene on a particular variable? If the causal relationships (e.g.~a causal graph) are known, it is possible to estimate the intervention distributions. In the absence of this domain knowledge, the causal structure must be discovered from the available observational data. However, observational data are often compatible with multiple causal graphs, making methods that commit to a single structure prone to overconfidence. A principled way to manage this structural uncertainty is via Bayesian inference, which averages over a posterior distribution on possible causal structures and functional mechanisms. Unfortunately, the number of causal structures grows super-exponentially with the number of nodes in the graph, making computations intractable. We propose to circumvent these challenges by using meta-learning to create an end-to-end model: the Model-Averaged Causal Estimation Transformer Neural Process (MACE-TNP). The model is trained to predict the Bayesian model-averaged interventional posterior distribution, and its end-to-end nature bypasses the need for expensive calculations. Empirically, we demonstrate that MACE-TNP outperforms strong Bayesian baselines. Our work establishes meta-learning as a flexible and scalable paradigm for approximating complex Bayesian causal inference, that can be scaled to increasingly challenging settings in the future.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 38 of its 68 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

38 of 68 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  3. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  4. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 471
  5. 2018Conditional Neural Processescited by 426
  6. 2021Transformers Can Do Bayesian Inferencecited by 360
  7. 2018Neural Processescited by 311
  8. 2020Differentiable Causal Discovery from Interventional Datacited by 218
  9. 2019Convolutional Conditional Neural Processescited by 142
  10. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 127
  11. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
  12. 2018Meta-Learning Probabilistic Inference For Predictioncited by 110
26 more
  1. 2021DiBS: Differentiable Bayesian Structure Learningcited by 106
  2. 2022Deep End-to-end Causal Inferencecited by 95
  3. 2022Amortized Inference for Causal Structure Learningcited by 90
  4. 2017Permutation-based Causal Inference Algorithms with Interventionscited by 78
  5. 2022Learning to Induce Causal Structurecited by 68
  6. 2013Understanding predictive information criteria for Bayesian modelscited by 52
  7. 2022Differentiable DAG Samplingcited by 47
  8. 2023Causal Discovery with Score Matching on Additive Models with Arbitrary Noisecited by 43
  9. 2013Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributionscited by 42
  10. 2022Active Bayesian Causal Inferencecited by 35
  11. 2015Partition MCMC for inference on acyclic digraphscited by 33
  12. 2020Towards Scalable Bayesian Learning of Causal DAGscited by 33
  13. 2022Latent Bottlenecked Attentive Neural Processescited by 26
  14. 2024A Meta-Learning Approach to Bayesian Causal Discoverycited by 22
  15. 2013A Bayesian Approach to Learning Causal Networkscited by 20
  16. 2024Standardizing Structural Causal Modelscited by 17
  17. 2013Gaussian Process Networkscited by 16
  18. 2024Translation Equivariant Transformer Neural Processescited by 13
  19. 2025Black Box Causal Inference: Effect Estimation via Meta Predictioncited by 13
  20. 2023Bivariate Causal Discovery using Bayesian Model Selectioncited by 12
  21. 2023Towards Bounding Causal Effects under Markov Equivalencecited by 8
  22. 2024Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orderscited by 6
  23. 2024Continuous Bayesian Model Selection for Multivariate Causal Discoverycited by 6
  24. 2024Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Datacited by 4
  25. 2024Bayesian Causal Inference with Gaussian Process Networkscited by 3
  26. 2024Transformer Neural Processes - Kernel Regressioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 12.

13 papers.

Sort:

By area:

  1. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 132
  2. 2026Use What You Know: Causal Foundation Models with Partial Graphscited by 7
  3. 2026Improving TabPFN's Synthetic Data Generation by Integrating Causal Structurecited by 3
  4. 2026Computational Identifiabilitycited by 2
  5. 2026PerturbPFN: Probing the Limits of Synthetic Priors in Drug Perturbation Modellingcited by 2
  6. 2026Foundation Models for Partial Causal Identificationcited by 2
  7. 2026Causal Foundation Modelscited by 1
  8. 2026PRIM: Meta-Learned Bayesian Root Cause Analysiscited by 0
  9. 2026Amortized Interventional Forecasting for Multivariate CIR Processescited by 0
  10. 2026CAFE: Counterfactual Prediction via Fast Posterior Estimationcited by 0
  11. 2026CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimescited by 0
  12. 2026Market Impact Estimation via Causal Foundation Modelscited by 0
1 more
  1. 2026Amortized Counterfactual Prediction with Graph Neural Networkscited by 0