mlsanityData to 7 Oct 2026

Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey

arXiv 2402.17944 ↗First posted 27 Feb 2024cs.CLcited by 157

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent breakthroughs in large language modeling have facilitated rigorous exploration of their application in diverse tasks related to tabular data modeling, such as prediction, tabular data synthesis, question answering, and table understanding. Each task presents unique challenges and opportunities. However, there is currently a lack of comprehensive review that summarizes and compares the key techniques, metrics, datasets, models, and optimization approaches in this research domain. This survey aims to address this gap by consolidating recent progress in these areas, offering a thorough survey and taxonomy of the datasets, metrics, and methodologies utilized. It identifies strengths, limitations, unexplored territories, and gaps in the existing literature, while providing some insights for future research directions in this vital and rapidly evolving field. It also provides relevant code and datasets references. Through this comprehensive review, we hope to provide interested readers with pertinent references and insightful perspectives, empowering them with the necessary tools and knowledge to effectively navigate and address the prevailing challenges in the field.

Key facts. Cited by 157 papers in the map (data through 7 Oct 2026). Builds on 162 of its 223 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

162 of 223 references.

References · 162

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  12. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2018Deep contextualized word representationscited by 2,285
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  7. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  10. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  11. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  12. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  13. 2020Efficient Transformers: A Surveycited by 784
  14. 2023Large Language Modelscited by 774
  15. 2016Wide & Deep Learning for Recommender Systemscited by 745
  16. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  17. 2019Modeling Tabular data using Conditional GANcited by 695
  18. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  19. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  20. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  21. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  22. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  23. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  24. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  25. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  26. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  27. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  28. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  29. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  30. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  31. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
  32. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  33. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  34. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
  35. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  36. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  37. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  38. 2022UL2: Unifying Language Learning Paradigmscited by 292

50 most cited of 162 All 162 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 39; 2025: 73; 2026: 45.

157 papers.

Sort:

By area:

  1. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 97
  2. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
  3. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 58
  4. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 55
  5. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  6. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  7. 2024CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solverscited by 40
  8. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  9. 2024Text2SQL is Not Enough: Unifying AI and Databases with TAGcited by 34
  10. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 34
  11. 2024Exploring Large Language Models for Feature Selection: A Data-centric Perspectivecited by 27
  12. 2025A Comprehensive Survey of Synthetic Tabular Data Generationcited by 27
38 more
  1. 2024LLM-Forest: Ensemble Learning of LLMs with Graph-Augmented Prompts for Data Imputationcited by 25
  2. 2024Evaluating language models as risk scorescited by 24
  3. 2024BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Taskscited by 19
  4. 2024Interpretable LLM-based Table Question Answeringcited by 18
  5. 2024Magneto: Combining Small and Large Language Models for Schema Matchingcited by 17
  6. 2024Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AIcited by 15
  7. 2024TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Modelscited by 15
  8. 2025TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fieldscited by 15
  9. 2025Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learningcited by 15
  10. 2025AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Researchcited by 12
  11. 2025On LLM-Enhanced Mixed-Type Data Imputation with High-Order Message Passingcited by 11
  12. 2025LLM Embeddings for Deep Learning on Tabular Datacited by 11
  13. 2025Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Frameworkcited by 10
  14. 2025Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoningcited by 10
  15. 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tablescited by 9
  16. 2025NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tablescited by 8
  17. 2025Joint Relational Database Generation via Graph-Conditional Diffusion Modelscited by 8
  18. 2024MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Datacited by 7
  19. 2025In-context Learning of Evolving Data Streams with Tabular Foundational Modelscited by 7
  20. 2025LLM Meeting Decision Trees on Tabular Datacited by 7
  21. 2024Scalable and Domain-General Abstractive Proposition Segmentationcited by 6
  22. 2024AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handlercited by 6
  23. 2024Enriching Tabular Data with Contextual LLM Embeddings: A Comprehensive Ablation Study for Ensemble Classifierscited by 6
  24. 2024Tabular Data Synthesis with Differential Privacy: A Surveycited by 6
  25. 2024Improving LLM Group Fairness on Tabular Data via In-Context Learningcited by 6
  26. 2025GLARE: Agentic Reasoning for Legal Judgment Predictioncited by 6
  27. 2025AECBench: A Hierarchical Benchmark for Knowledge Evaluation of Large Language Models in the AEC Fieldcited by 6
  28. 2025Efficient numeracy in language models through single-token number embeddingscited by 6
  29. 2025SheetBrain: A Neuro-Symbolic Agent for Accurate Reasoning over Complex and Large Spreadsheetscited by 6
  30. 2025Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understandingcited by 6
  31. 2025Orion-Bix: Bi-Axial Attention for Tabular In-Context Learningcited by 6
  32. 2024Quantifying Prediction Consistency Under Fine-Tuning Multiplicity in Tabular LLMscited by 5
  33. 2024Gradient Boosting Trees and Large Language Models for Tabular Data Few-Shot Learningcited by 5
  34. 2025Position: Foundation Models for Tabular Data within Systemic Contexts Need Groundingcited by 5
  35. 2025STRuCT-LLM: Unifying Tabular and Graph Reasoning with Reinforcement Learning for Semantic Parsingcited by 5
  36. 2025Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Modelscited by 5
  37. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  38. 2026FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principlescited by 5

50 most cited of 157 All 157 (JSON)