mlsanityData to 7 Oct 2026

Table-GPT: Table-tuned GPT for Diverse Table Tasks

arXiv 2310.09263 ↗First posted 13 Oct 2023cs.CL cs.AI cs.DBcited by 69

Not on the map. Why?

Abstract (arXiv)

Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on one-dimensional natural-language texts, whereas relational tables are two-dimensional objects. In this work, we propose a new "table-tuning" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better table-understanding capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong generalizability, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Builds on 37 of its 71 references that are in the map. First posted on arXiv 13 Oct 2023.

Builds on

37 of 71 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
25 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  4. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  7. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  8. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  9. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  10. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  11. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  12. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  13. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  14. 2022Can Foundation Models Wrangle Your Data?cited by 173
  15. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  16. 2023Self-Alignment with Instruction Backtranslationcited by 149
  17. 2020Deep Entity Matching with Pre-Trained Language Modelscited by 143
  18. 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
  19. 2019Sherlock: A Deep Learning Approach to Semantic Data Type Detectioncited by 73
  20. 2021Annotating Columns with Pre-trained Language Modelscited by 37
  21. 2020Valentine: Evaluating Matching Techniques for Dataset Discoverycited by 34
  22. 2023Using ChatGPT for Entity Matchingcited by 22
  23. 2023Column Type Annotation using ChatGPTcited by 18
  24. 2023Auto-BI: Automatically Build BI-Models Leveraging Local Join Prediction and Global Schema Graphcited by 7
  25. 2018Wrangling Messy CSV Files by Detecting Row and Type Patternscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 36; 2025: 28; 2026: 2.

69 papers.

Sort:

By area:

  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  2. 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 136
  3. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
  4. 2024Large Language Models Meet NLP: A Surveycited by 102
  5. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  6. 2024TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarioscited by 76
  7. 2024Large Language Model for Table Processing: A Surveycited by 52
  8. 2023Tabular Representation, Noisy Operators, and Impacts on Table Structure Understanding Tasks in LLMscited by 49
  9. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  10. 2024Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extractioncited by 33
  11. 2024Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Datacited by 24
  12. 2024ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Contextcited by 23
38 more
  1. 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 23
  2. 2023Jellyfish: A Large Language Model for Data Preprocessingcited by 20
  3. 2025Sequential Diffusion Language Modelscited by 19
  4. 2024UQE: A Query Engine for Unstructured Databasescited by 18
  5. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
  6. 2024SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Modelscited by 16
  7. 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 15
  8. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 15
  9. 2025Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoningcited by 14
  10. 2025MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmarkcited by 13
  11. 2024Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretrainingcited by 12
  12. 2024Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representationscited by 12
  13. 2025ST-Raptor: LLM-Powered Semi-Structured Table Question Answeringcited by 11
  14. 2024Causal Agent based on Large Language Modelcited by 10
  15. 2025Rethinking Table Instruction Tuningcited by 10
  16. 2024SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQAcited by 9
  17. 2024Evidence Contextualization and Counterfactual Attribution for Conversational QA over Heterogeneous Data with RAG Systemscited by 8
  18. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  19. 2024Revolutionizing Database Q&A with Large Language Models: Comprehensive Benchmark and Evaluationcited by 7
  20. 2025Advancing the Scientific Method with Large Language Models: From Hypothesis to Discoverycited by 7
  21. 2025Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Modelscited by 7
  22. 2025Nile-Chat: Egyptian Language Models for Arabic and Latin Scriptscited by 7
  23. 2024Data Analysis in the Era of Generative AIcited by 6
  24. 2025Map&Make: Schema Guided Text to Table Generationcited by 6
  25. 2024DataFrame QA: A Universal LLM Framework on DataFrame Question Answering Without Data Exposurecited by 5
  26. 2024Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategiescited by 5
  27. 20247B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancementcited by 5
  28. 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoningcited by 5
  29. 2024Had enough of experts? Quantitative knowledge retrieval from large language modelscited by 4
  30. 2024Vision Language Models for Spreadsheet Understanding: Challenges and Opportunitiescited by 4
  31. 2025TableDreamer: Progressive and Weakness-guided Data Synthesis from Scratch for Table Instruction Tuningcited by 4
  32. 2024Graph Neural Network Approach to Semantic Type Detection in Tablescited by 3
  33. 2024Generating Tables from the Parametric Knowledge of Language Modelscited by 3
  34. 2024DataTales: A Benchmark for Real-World Intelligent Data Narrationcited by 3
  35. 2025Empowering Tabular Data Preparation with Language Models: Why and How?cited by 3
  36. 2024Evaluating LLMs on Entity Disambiguation in Tablescited by 2
  37. 20242D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Modelscited by 2
  38. 2024BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Datacited by 2

50 most cited of 69 All 69 (JSON)