mlsanityData to 14 Oct 2026

TabLLM: Few-shot Classification of Tabular Data with Large Language Models

arXiv 2210.10723 ↗First posted 19 Oct 2022cs.CL cs.AIcited by 277

Not on the map. Why?

Abstract (arXiv)

We study the application of large language models to zero-shot and few-shot classification of tabular data. We prompt the large language model with a serialization of the tabular data to a natural-language string, together with a short description of the classification problem. In the few-shot setting, we fine-tune the large language model using some labeled examples. We evaluate several serialization methods including templates, table-to-text models, and large language models. Despite its simplicity, we find that this technique outperforms prior deep-learning-based tabular classification methods on several benchmark datasets. In most cases, even zero-shot classification obtains non-trivial performance, illustrating the method's ability to exploit prior knowledge encoded in large language models. Unlike many deep learning methods for tabular datasets, this approach is also competitive with strong traditional baselines like gradient-boosted trees, especially in the very-few-shot setting.

Key facts. Cited by 277 papers in the map (data through 14 Oct 2026). Builds on 36 of its 63 references that are in the map. First posted on arXiv 19 Oct 2022.

Builds on

36 of 63 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  9. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  10. 2021GPT Understands, Toocited by 716
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  12. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
24 more
  1. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  2. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  3. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  4. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  5. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
  6. 2021True Few-Shot Learning with Language Modelscited by 344
  7. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  8. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  9. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  10. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  11. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 203
  12. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 186
  13. 2022Can Foundation Models Wrangle Your Data?cited by 175
  14. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 156
  15. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  16. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 148
  17. 2020Deep Entity Matching with Pre-Trained Language Modelscited by 144
  18. 2021A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Modelscited by 101
  19. 2022Prototypical Verbalizer for Prompt-based Few-shot Tuningcited by 53
  20. 2022Transfer Learning with Deep Tabular Modelscited by 52
  21. 2017Improving Palliative Care with Deep Learningcited by 28
  22. 2022Co-training Improves Prompt-based Learning for Large Language Modelscited by 24
  23. 2020Deep Contextual Clinical Prediction with Reverse Distillationcited by 16
  24. 2016Endgame Analysis of Dou Shou Qicited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 48; 2024: 70; 2025: 89; 2026: 70.

277 papers.

Sort:

By area:

  1. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
  2. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  3. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  4. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
  5. 2023GPT-NER: Named Entity Recognition via Large Language Modelscited by 160
  6. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  7. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 142
  8. 2024Large Language Models to Enhance Bayesian Optimizationcited by 135
  9. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  10. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  11. 2024Large Language Models for Time Series: A Surveycited by 114
  12. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 113
38 more
  1. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  2. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 77
  3. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 76
  4. 2024Large Scale Transfer Learning for Tabular Data via Language Modelingcited by 76
  5. 2023Meta-in-context learning in large language modelscited by 59
  6. 2023TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPTcited by 59
  7. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 59
  8. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  9. 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
  10. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  11. 2023Multimodal LLMs for health grounded in individual-specific datacited by 49
  12. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoningcited by 49
  13. 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 47
  14. 2023BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biologycited by 43
  15. 2025A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilitiescited by 41
  16. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 38
  17. 2023Language Models are Few-shot Learners for Prognostic Predictioncited by 37
  18. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 36
  19. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  20. 2023Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous drivingcited by 30
  21. 2023TABLET: Learning From Instructions For Tabular Datacited by 28
  22. 2023ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Predictioncited by 27
  23. 2024Evaluating language models as risk scorescited by 27
  24. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 26
  25. 2025Clinical ModernBERT: An efficient and long context encoder for biomedical textcited by 26
  26. 2023LLM4Vis: Explainable Visualization Recommendation using ChatGPTcited by 25
  27. 2024Privacy Preserving Prompt Engineering: A Surveycited by 25
  28. 2023MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinementcited by 24
  29. 2024Generative AI for Visualization: State of the Art and Future Directionscited by 23
  30. 2024Emergency Department Decision Support using Clinical Pseudo-notescited by 22
  31. 2023Column Type Annotation using ChatGPTcited by 19
  32. 2023MotherNet: Fast Training and Inference via Hyper-Network Transformerscited by 19
  33. 2024Elephants Never Forget: Testing Language Models for Memorization of Tabular Datacited by 19
  34. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  35. 2025Table Foundation Models: on knowledge pre-training for tabular learningcited by 19
  36. 2025Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learningcited by 19
  37. 2024GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understandingcited by 18
  38. 2024A Survey on Deep Tabular Learningcited by 18

50 most cited of 277 All 277 (JSON)