mlsanityData to 7 Oct 2026

TableLlama: Towards Open Large Generalist Models for Tables

arXiv 2311.09206 ↗First posted 15 Nov 2023cs.CL cs.AI cs.DBcited by 136

Not on the map. Why?

Cited by 32 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves comparable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 5-44 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model's generalizability. We open-source our dataset and trained model to boost future work on developing open generalist models for tables.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). Builds on 29 of its 57 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

29 of 57 references.

References · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  7. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  8. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  9. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  10. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  11. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  12. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
17 more
  1. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  2. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  3. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  4. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  5. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  6. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  7. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  8. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  9. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  10. 2017Key-Value Retrieval Networks for Task-Oriented Dialoguecited by 137
  11. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  12. 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
  13. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  14. 2019Sherlock: A Deep Learning Approach to Semantic Data Type Detectioncited by 73
  15. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  16. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  17. 2017EntiTables: Smart Assistance for Entity-Focused Tablescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 45; 2025: 54; 2026: 33.

136 papers.

Sort:

By area:

  1. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  2. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  3. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 124
  4. 2024Large Language Models Meet NLP: A Surveycited by 102
  5. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  6. 2024TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarioscited by 76
  7. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
  8. 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Datacited by 61
  9. 2024Large Language Model for Table Processing: A Surveycited by 52
  10. 2024Multimodal Table Understandingcited by 43
  11. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMscited by 40
  12. 2024TableRAG: Million-Token Table Understanding with Language Modelscited by 40
38 more
  1. 2024SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literaturecited by 37
  2. 2025Table-R1: Region-based Reinforcement Learning for Table Understandingcited by 35
  3. 2023Rethinking Tabular Data Understanding with Large Language Modelscited by 34
  4. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  5. 2024A Survey of Table Reasoning with Large Language Modelscited by 29
  6. 2024StructLM: Towards Building Generalist Models for Structured Knowledge Groundingcited by 29
  7. 2024ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Contextcited by 23
  8. 2025Table-R1: Inference-Time Scaling for Table Reasoningcited by 23
  9. 2023Jellyfish: A Large Language Model for Data Preprocessingcited by 20
  10. 2025TableMaster: A Recipe to Advance Table Understanding with Language Modelscited by 18
  11. 2024TQA-Bench: Evaluating LLMs for Multi-Table Question Answeringcited by 17
  12. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
  13. 2024NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalizationcited by 16
  14. 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 15
  15. 2025Table Foundation Models: on knowledge pre-training for tabular learningcited by 15
  16. 2025Does Table Source Matter? Benchmarking and Improving Multimodal Scientific Table Understanding and Reasoningcited by 14
  17. 2025TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answeringcited by 14
  18. 2023SUQL: Conversational Search over Structured and Unstructured Data with Large Language Modelscited by 13
  19. 2024PoTable: Towards Systematic Thinking via Plan-then-Execute Stage Reasoning on Tablescited by 13
  20. 2024Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answeringcited by 13
  21. 2025TabPFN-Wide: Continued Pre-Training for Extreme Feature Countscited by 13
  22. 2024Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretrainingcited by 12
  23. 2024Efficient Prompting for LLM-based Generative Internet of Thingscited by 12
  24. 2024MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoningcited by 12
  25. 2025RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysiscited by 12
  26. 2025HyperG: Hypergraph-Enhanced LLMs for Structured Knowledgecited by 11
  27. 2025ST-Raptor: LLM-Powered Semi-Structured Table Question Answeringcited by 11
  28. 2024cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Paperscited by 10
  29. 2024Causal Agent based on Large Language Modelcited by 10
  30. 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoningcited by 10
  31. 2025RoT: Enhancing Table Reasoning with Iterative Row-Wise Traversalscited by 10
  32. 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
  33. 2024Uncovering Limitations of Large Language Models in Information Seeking from Tablescited by 9
  34. 2025MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Spacecited by 9
  35. 2026Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMscited by 9
  36. 2024FLEXTAF: Enhancing Table Reasoning with Flexible Tabular Formatscited by 8
  37. 2025NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tablescited by 8
  38. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8

50 most cited of 136 All 136 (JSON)