mlsanityData to 14 Oct 2026

CausalPFN: Amortized Causal Effect Estimation via In-Context Learning

arXiv 2506.07918 ↗First posted 9 Jun 2025cs.LG stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

Causal effect estimation from observational data is fundamental across various applications. However, selecting an appropriate estimator from dozens of specialized methods demands substantial manual effort and domain expertise. We present CausalPFN, a single transformer that amortizes this workflow: trained once on a large library of simulated data-generating processes that satisfy ignorability, it infers causal effects for new observational datasets out of the box. CausalPFN combines ideas from Bayesian causal inference with the large-scale training protocol of prior-fitted networks (PFNs), learning to map raw observations directly to causal effects without any task-specific adjustment. Our approach achieves superior average performance on heterogeneous and average treatment effect estimation benchmarks (IHDP, Lalonde, ACIC). Moreover, it shows competitive performance for real-world policy making on uplift modeling tasks. CausalPFN provides calibrated uncertainty estimates to support reliable decision-making based on Bayesian principles. This ready-to-use model requires no further training or tuning and takes a step toward automated causal inference (https://github.com/vdblm/CausalPFN/).

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 75 of its 123 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

75 of 123 references.

References · 75

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  6. 2022In-context Learning and Induction Headscited by 964
  7. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  8. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
  9. 2022Transformers learn in-context by gradient descentcited by 732
  10. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  12. 2017Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learningcited by 478
38 more
  1. 2018Conditional Neural Processescited by 426
  2. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 410
  3. 2021Transformers Can Do Bayesian Inferencecited by 360
  4. 2016Generalized Random Forestscited by 312
  5. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  6. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
  7. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  8. 2019Adapting Neural Networks for the Estimation of Treatment Effectscited by 287
  9. 2019Attentive Neural Processescited by 283
  10. 2020Query-Key Normalization for Transformerscited by 280
  11. 2013Causal Discovery with Continuous Additive Noise Modelscited by 255
  12. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  13. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  14. 2017OpenML Benchmarking Suitescited by 178
  15. 2024The Road Less Scheduledcited by 163
  16. 2017Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processescited by 158
  17. 2019Orthogonal Statistical Learningcited by 151
  18. 2021Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithmscited by 144
  19. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  20. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  21. 2017Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competitioncited by 106
  22. 2019FLAML: A Fast and Lightweight AutoML Librarycited by 106
  23. 2023Uncovering mesa-optimization algorithms in Transformerscited by 100
  24. 2020Causal Autoregressive Flowscited by 99
  25. 2021The Causal-Neural Connection: Expressiveness, Learnability, and Inferencecited by 98
  26. 2022Amortized Inference for Causal Structure Learningcited by 90
  27. 2021Discussion on "Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects" by Hahn, Murray and Carvalhocited by 77
  28. 2021On Inductive Biases for Heterogeneous Treatment Effect Estimationcited by 75
  29. 2022Learning to Induce Causal Structurecited by 68
  30. 2018Improving Regression Performance with Distributional Lossescited by 67
  31. 2023Meta-in-context learning in large language modelscited by 59
  32. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 59
  33. 2020Identifying Causal-Effect Inference Failure with Uncertainty-Aware Modelscited by 57
  34. 2022Neural Causal Models for Counterfactual Identification and Estimationcited by 57
  35. 2022AMLB: an AutoML Benchmarkcited by 55
  36. 2025Do-PFN: In-Context Learning for Causal Effect Estimationcited by 51
  37. 2020RealCause: Realistic Causal Inference Benchmarkingcited by 46
  38. 2023Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Modelscited by 45

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 45.

49 papers.

Sort:

By area:

  1. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 132
  2. 2025Foundation Models for Causal Inference via Prior-Data Fitted Networkscited by 32
  3. 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 23
  4. 2026TabPFN-3: Technical Reportcited by 17
  5. 2026Relational In-Context Learning via Synthetic Pre-training with Structural Priorcited by 14
  6. 2026From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detectioncited by 9
  7. 2026Use What You Know: Causal Foundation Models with Partial Graphscited by 7
  8. 2026Causal Foundation Models with Continuous Treatmentscited by 4
  9. 2026MapPFN: Learning Causal Perturbation Maps in Contextcited by 3
  10. 2026Improving TabPFN's Synthetic Data Generation by Integrating Causal Structurecited by 3
  11. 2026Amortizing Causal Sensitivity Analysis via Prior Data-Fitted Networkscited by 3
  12. 2026IV-ICL: Bounding Causal Effects with Instrumental Variables via In-Context Learningcited by 3
37 more
  1. 2026SurvivalPFN: Amortizing Survival Prediction via In-Context Bayesian Inferencecited by 3
  2. 2026FoundCause: Causal Discovery with Latent Confounders from Observational Datacited by 3
  3. 2025Causal-EPIG: A Prediction-Oriented Active Learning Framework for CATE Estimationcited by 2
  4. 2025Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspectivecited by 2
  5. 2026Your Autoregressive Model Already Reveals the Causal Graphcited by 2
  6. 2026Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Modelscited by 2
  7. 2026Computational Identifiabilitycited by 2
  8. 2026A Causal Foundation Model for Structure and Outcome Predictioncited by 2
  9. 2026CDFM: Towards a General-Purpose Causal Discovery Foundation Modelcited by 2
  10. 2026PerturbPFN: Probing the Limits of Synthetic Priors in Drug Perturbation Modellingcited by 2
  11. 2026Understanding the Surprising Generalization Properties of Tabular Foundation Modelscited by 2
  12. 2026Foundation Models for Partial Causal Identificationcited by 2
  13. 2026Observationally Informed Adaptive Causal Experimental Designcited by 1
  14. 2026Learning Causal Orderings for In-Context Tabular Predictioncited by 1
  15. 2026Causal Longitudinal Prior-Fitted Networks for Counterfactual Outcome Predictioncited by 1
  16. 2026Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignmentcited by 1
  17. 2026Causal Foundation Modelscited by 1
  18. 2026Nearly Optimal Bayesian Inference for Structural Missingnesscited by 0
  19. 2026Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Datacited by 0
  20. 2026TabCF: Distributional Control Function Estimation with Tabular Foundation Modelscited by 0
  21. 2026In-context learning to predict critical transitions in dynamical systemscited by 0
  22. 2026Modular Multimodal Classification Without Fine-Tuning: A Simple Compositional Approachcited by 0
  23. 2026The Good, the Bad, and the Ugly of Markov Boundary for Tabular Predictioncited by 0
  24. 2026FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Taskscited by 0
  25. 2026TabPATE: Differentially Private Tabular In-Context Learning Without Public Datacited by 0
  26. 2026Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?cited by 0
  27. 2026Amortized Interventional Forecasting for Multivariate CIR Processescited by 0
  28. 2026Training Fair Tabular Foundation Modelscited by 0
  29. 2026When and Why LLM Causal Priors Help: Closed-Loop Prior Selection for Amortized Causal Inferencecited by 0
  30. 2026Splitting the Difference: Interpretable Causal Forests for Treatment Effect Heterogeneity and Biascited by 0
  31. 2026CAFE: Counterfactual Prediction via Fast Posterior Estimationcited by 0
  32. 2026What You Observe Determines How You Identify Causal Effects: Evaluating Causal Models across Observational Viewscited by 0
  33. 2026CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimescited by 0
  34. 2026Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysiscited by 0
  35. 2026Beyond Distributional Fidelity: Causal-Penalized Diffusion for Synthetic Tabular Datacited by 0
  36. 2026Market Impact Estimation via Causal Foundation Modelscited by 0
  37. 2026How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?cited by 0