mlsanityData to 14 Oct 2026

TabFM: A Zero-Shot Foundation Model for Tabular Data

arXiv 2609.37959 ↗First posted 29 Sep 2026cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. TabFM produces calibrated zero-shot predictions in a single forward pass without task-specific tuning. Trained entirely on synthetic tables generated from structural causal models, TabFM learns general tabular representations that transfer zero-shot to real-world tasks. Across all 51 benchmark datasets in TabArena (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines. Two extensions over the same frozen weights improve performance further on both tracks: multi-view feature expansion with ensembling and post-hoc calibration (TabFM+), and LLM-guided, dataset-specific data processing and feature engineering (TabFM-Auto).

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

30 of 46 references.

References · 30

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2019Root Mean Square Layer Normalizationcited by 1,480
  6. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  7. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  8. 2022Transformers learn in-context by gradient descentcited by 732
  9. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  10. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  11. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  12. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
18 more
  1. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  2. 2021Transformers Can Do Bayesian Inferencecited by 360
  3. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  4. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  5. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  6. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  7. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  8. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
  9. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 144
  10. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  11. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  12. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  13. 2023Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regressioncited by 49
  14. 2025Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Datacited by 41
  15. 2024Pre-trained Large Language Models Use Fourier Features to Compute Additioncited by 39
  16. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  17. 2025FoNE: Precise Single-Token Number Embeddings via Fourier Featurescited by 22
  18. 2026Convergent Evolution: How Different Language Models Learn Similar Number Representationscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026RelICL: Training-free Relational Learning with Tabular Foundation Modelscited by 0
  2. 2026Tabular Foundation Models Are Effectively Shallowcited by 0
  3. 2026How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?cited by 0
  4. 2026Recurrent In-Context Learning for Tabular Foundation Modelscited by 0
  5. 2026When Repeated Measurements Hurt Tabular Foundation Modelscited by 0
  6. 2026OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learningcited by 0