# ACTIVA: Amortized Causal Effect Estimation via Transformer-based Variational Autoencoder

arXiv: https://arxiv.org/abs/2503.01290 · HTML: https://mlsanity.com/paper/2503.01290/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning](https://mlsanity.com/paper/2106.02584/) (2021)
- [DiBS: Differentiable Bayesian Structure Learning](https://mlsanity.com/paper/2105.11839/) (2021)
- [Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data](https://mlsanity.com/paper/2006.10833/) (2020)
- [Amortized Inference for Causal Structure Learning](https://mlsanity.com/paper/2205.12934/) (2022)
- [Diffusion Causal Models for Counterfactual Estimation](https://mlsanity.com/paper/2202.10166/) (2022)
- [Relating Graph Neural Networks to Structural Causal Models](https://mlsanity.com/paper/2109.04173/) (2021)
- [Supervised Whole DAG Causal Discovery](https://mlsanity.com/paper/2006.04697/) (2020)
- [Zero-shot causal learning](https://mlsanity.com/paper/2301.12292/) (2023)
- [Demystifying amortized causal discovery with transformers](https://mlsanity.com/paper/2405.16924/) (2024)
- [Amortized Active Causal Induction with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.16718/) (2024)
- [Modular Learning of Deep Causal Generative Models for High-dimensional Causal Inference](https://mlsanity.com/paper/2401.01426/) (2024)
- [β-Intact-VAE: Identifying and Estimating Causal Effects under Limited Overlap](https://mlsanity.com/paper/2110.05225/) (2021)
- [Learning Structural Causal Models through Deep Generative Models: Methods, Guarantees, and Challenges](https://mlsanity.com/paper/2405.05025/) (2024)
- [Targeted VAE: Variational and Targeted Learning for Causal Inference](https://mlsanity.com/paper/2009.13472/) (2020)
- [CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/2401.16974/) (2024)
- [Causal Inference for Banking Finance and Insurance A Survey](https://mlsanity.com/paper/2307.16427/) (2023)
- [CausalPlayground: Addressing Data-Generation Requirements in Cutting-Edge Causality Research](https://mlsanity.com/paper/2405.13092/) (2024)
- [CMVAE: Causal Meta VAE for Unsupervised Meta-Learning](https://mlsanity.com/paper/2302.09731/) (2023)
- [Estimating Joint Interventional Distributions from Marginal Interventional Data](https://mlsanity.com/paper/2409.01794/) (2024)

## Built on by (most cited first, 1 of 1)

- [TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models](https://mlsanity.com/paper/2511.08667/) (2025)

## Abstract (arXiv)

Predicting post-intervention distributions from observational data is central to many scientific and decision-making problems, but remains challenging due to causal ambiguity, restrictive modeling assumptions, and the lack of amortization across tasks. We introduce ACTIVA, a transformer-based conditional variational autoencoder for amortized estimation of full interventional distributions from observational data and intervention queries. ACTIVA learns a conditional latent prior that supports zero-shot inference by amortizing causal knowledge across diverse training tasks. We provide a consistency result showing that, under idealized conditions, ACTIVA's learning objective targets a mixture over the interventional distributions of causal models that are observationally compatible with the input. Empirically, on synthetic datasets and biologically realistic gene-expression simulations, ACTIVA substantially outperforms a correlational baseline, reduces spurious non-descendant effects, and achieves competitive performance relative to strong amortized baselines. Our results show that ACTIVA is a promising approach for estimating interventional distributions from observational data.