# What exactly has TabPFN learned to do?

arXiv: https://arxiv.org/abs/2502.08978 · HTML: https://mlsanity.com/paper/2502.08978/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2025.

## Built on by (most cited first, 5 of 5)

- [CausalPFN: Amortized Causal Effect Estimation via In-Context Learning](https://mlsanity.com/paper/2506.07918/) (2025)
- [Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models](https://mlsanity.com/paper/2605.06510/) (2026)
- [From Tables to Signals: Revealing Spectral Adaptivity in TabPFN](https://mlsanity.com/paper/2511.18278/) (2025)
- [FoMo X: Modular Explainability Signals for Outlier Detection Foundation Models](https://mlsanity.com/paper/2603.17570/) (2026)
- [Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit](https://mlsanity.com/paper/2609.02766/) (2026)

## Abstract (arXiv)

TabPFN [Hollmann et al., 2023], a Transformer model pretrained to perform in-context learning on fresh tabular classification problems, was presented at the last ICLR conference. To better understand its behavior, we treat it as a black-box function approximator generator and observe its generated function approximations on a varied selection of training datasets. Exploring its learned inductive biases in this manner, we observe behavior that is at turns either brilliant or baffling. We conclude this post with thoughts on how these results might inform the development, evaluation, and application of prior-data fitted networks (PFNs) in the future.