# Generalization Can Emerge in Tabular Foundation Models From a Single Table

arXiv: https://arxiv.org/abs/2511.09665 · HTML: https://mlsanity.com/paper/2511.09665/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2025.

## Built on by (most cited first, 4 of 4)

- [Causal Foundation Models with Continuous Treatments](https://mlsanity.com/paper/2605.15133/) (2026)
- [Enhancing Tabular Learners with Context-Aware Semantic Embeddings](https://mlsanity.com/paper/2608.03565/) (2026)
- [FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Tasks](https://mlsanity.com/paper/2606.30336/) (2026)
- [Topological Signatures of Context-Level Reliability in TabPFN](https://mlsanity.com/paper/2607.17962/) (2026)

## Abstract (arXiv)

Deep tabular modelling increasingly relies on in-context learning where, during inference, a model receives a set of (x,y) pairs as context and predicts labels for new inputs without weight updates. We challenge the prevailing view that broad generalization here requires pre-training on large synthetic corpora (e.g., TabPFN priors) or a large collection of real data (e.g., TabDPT training datasets), discovering that a relatively small amount of data suffices for generalization. We find that simple self-supervised pre-training on just a single real table can produce surprisingly strong transfer across heterogeneous benchmarks. By systematically pre-training and evaluating on many diverse datasets, we analyze what aspects of the data are most important for building a Tabular Foundation Model (TFM) generalizing across domains. We then connect this to the pre-training procedure shared by most TFMs and show that the number and quality of tasks one can construct from a dataset is key to downstream performance.