# Does TabPFN Understand Causal Structures?

arXiv: https://arxiv.org/abs/2511.07236 · HTML: https://mlsanity.com/paper/2511.07236/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Amortized Inference for Causal Structure Learning](https://mlsanity.com/paper/2205.12934/) (2022)
- [Permutation-based Causal Inference Algorithms with Interventions](https://mlsanity.com/paper/1705.10220/) (2017)
- [Learning to Induce Causal Structure](https://mlsanity.com/paper/2204.04875/) (2022)
- [Do-PFN: In-Context Learning for Causal Effect Estimation](https://mlsanity.com/paper/2506.06039/) (2025)
- [TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks](https://mlsanity.com/paper/2402.11137/) (2024)
- [Scaling structural learning with NO-BEARS to infer causal transcriptome networks](https://mlsanity.com/paper/1911.00081/) (2019)
- [Foundation Models for Causal Inference via Prior-Data Fitted Networks](https://mlsanity.com/paper/2506.10914/) (2025)
- [A Meta-Learning Approach to Bayesian Causal Discovery](https://mlsanity.com/paper/2412.16577/) (2024)
- [Recursive Causal Discovery](https://mlsanity.com/paper/2403.09300/) (2024)
- [Early Stopping Tabular In-Context Learning](https://mlsanity.com/paper/2506.21387/) (2025)
- [FairPFN: Transformers Can do Counterfactual Fairness](https://mlsanity.com/paper/2407.05732/) (2024)

## Built on by (most cited first, 5 of 5)

- [Noise Immunity in In-Context Tabular Learning: An Empirical Robustness Analysis of TabPFN's Attention Mechanisms](https://mlsanity.com/paper/2604.04868/) (2026)
- [Arrow: A Foundation Model for Causal Discovery](https://mlsanity.com/paper/2605.07204/) (2026)
- [CDFM: Towards a General-Purpose Causal Discovery Foundation Model](https://mlsanity.com/paper/2607.11508/) (2026)
- [Topological Signatures of Context-Level Reliability in TabPFN](https://mlsanity.com/paper/2607.17962/) (2026)
- [Characterizing the Separability Gap in Prior-Data Fitted Networks](https://mlsanity.com/paper/2610.14989/) (2026)

## Abstract (arXiv)

Causal discovery is fundamental for multiple scientific domains, yet extracting causal information from real world data remains a significant challenge. Given the recent success on real data, we investigate whether TabPFN, a transformer-based tabular foundation model pre-trained on synthetic datasets generated from structural causal models, encodes causal information in its internal representations. We develop an adapter framework using a learnable decoder and causal tokens that extract causal signals from TabPFN's frozen embeddings and decode them into adjacency matrices for causal discovery. Our evaluations demonstrate that TabPFN's embeddings contain causal information, outperforming several traditional causal discovery algorithms, with such causal information being concentrated in mid-range layers. These findings establish a new direction for interpretable and adaptable foundation models and demonstrate the potential for leveraging pre-trained tabular models for causal discovery.