mlsanityData to 14 Oct 2026

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

arXiv 2207.01848 ↗First posted 5 Jul 2022cs.LG stat.MLcited by 674

Not on the map. Why?

Abstract (arXiv)

We present TabPFN, a trained Transformer that can do supervised classification for small tabular datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods. TabPFN performs in-context learning (ICL), it learns to make predictions using sequences of labeled examples (x, f(x)) given in the input, without requiring further parameter updates. TabPFN is fully entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. This prior incorporates ideas from causal reasoning: It entails a large space of structural causal models with a preference for simple structures. On the 18 datasets in the OpenML-CC18 suite that contain up to 1 000 training data points, up to 100 purely numerical features without missing values, and up to 10 classes, we show that our method clearly outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 230× speedup. This increases to a 5 700× speedup when using a GPU. We also validate these results on an additional 67 small numerical datasets from OpenML. We provide all our code, the trained TabPFN, an interactive browser demo and a Colab notebook at https://github.com/automl/TabPFN.

Key facts. Cited by 674 papers in the map (data through 14 Oct 2026). Builds on 32 of its 73 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

32 of 73 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2020Longformer: The Long-Document Transformercited by 3,413
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  8. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  9. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  10. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  11. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  12. 2014OpenML: networked science in machine learningcited by 526
20 more
  1. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  2. 2017Progressive Neural Architecture Searchcited by 487
  3. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  4. 2021Transformers Can Do Bayesian Inferencecited by 360
  5. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  6. 2017OpenML Benchmarking Suitescited by 178
  7. 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174
  8. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  9. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 150
  10. 2018Anchor regression: heterogeneous data meets causalitycited by 118
  11. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
  12. 2022Amortized Inference for Causal Structure Learningcited by 90
  13. 2022Learning to Induce Causal Structurecited by 68
  14. 2019OpenML-Python: an extensible Python API for OpenMLcited by 63
  15. 2020Neural Ensemble Search for Uncertainty Estimation and Dataset Shiftcited by 56
  16. 2021MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanismscited by 51
  17. 2021Variational Causal Networks: Approximate Bayesian Inference over Causal Structurescited by 43
  18. 2020CASTLE: Regularization via Auxiliary Causal Graph Discoverycited by 41
  19. 2019Causal Regularizationcited by 21
  20. 2020From Probability to Consilience: How Explanatory Values Implement Bayesian Reasoningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 47; 2024: 98; 2025: 184; 2026: 340.

674 papers.

Sort:

By area:

  1. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  2. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  3. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  4. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  5. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  6. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  7. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 132
  8. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  9. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
  10. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  11. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 106
  12. 2022General-Purpose In-Context Learning by Meta-Learning Transformerscited by 105
38 more
  1. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  2. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 96
  3. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 87
  4. 2023XTab: Cross-table Pretraining for Tabular Transformerscited by 81
  5. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  6. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 77
  7. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 76
  8. 2024Large Scale Transfer Learning for Tabular Data via Language Modelingcited by 76
  9. 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 74
  10. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 73
  11. 2023Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Datacited by 69
  12. 2023Statistical Foundations of Prior-Data Fitted Networkscited by 65
  13. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  14. 2025From Tables to Time: Extending TabPFN-v2 to Time Series Forecastingcited by 60
  15. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 59
  16. 2024Making Pre-trained Language Models Great on Tabular Predictioncited by 57
  17. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  18. 2024TALENT: A Tabular Analytics and Learning Toolboxcited by 52
  19. 2025Do-PFN: In-Context Learning for Causal Effect Estimationcited by 51
  20. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 49
  21. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 48
  22. 2024TuneTables: Context Optimization for Scalable Prior-Data Fitted Networkscited by 47
  23. 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 47
  24. 2024Is Mamba Capable of In-Context Learning?cited by 45
  25. 2023Modeling rapid language learning by distilling Bayesian priors into artificial neural networkscited by 42
  26. 2025A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilitiescited by 41
  27. 2025Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Datacited by 41
  28. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 39
  29. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 38
  30. 2023Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networkscited by 37
  31. 2025TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problemscited by 36
  32. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  33. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  34. 2025Turning Tabular Foundation Models into Graph Foundation Modelscited by 35
  35. 2023ExcelFormer: A neural network surpassing GBDTs on tabular datacited by 34
  36. 2023Cross-Modal Fine-Tuning: Align then Refinecited by 34
  37. 2024OmniPred: Language Models as Universal Regressorscited by 34
  38. 2025ConTextTab: A Semantics-Aware Tabular In-Context Learnercited by 34

50 most cited of 674 All 674 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026TabPFN-3.5: Technical Report
  2. 2026TabSCM: A practical Framework for Generating Realistic Tabular Data
  3. 2026TabFORGE: A Tabular Foundation Model for Generative Modelling
  4. 2026LAB-Tab: LLM-Augmented Bayesian Network Adaptation for Few-Shot Tabular Generation
  5. 2026TabGift: A Context-Efficient Foundation Model for Tabular Data Generation
  6. 2026TabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery
  7. 2026TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks
  8. 2025TabNSA: Native Sparse Attention for Efficient Tabular Data Learning