mlsanityData to 7 Oct 2026

Table-To-Text generation and pre-training with TabT5

arXiv 2210.09162 ↗First posted 17 Oct 2022cs.CL cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Encoder-only transformer models have been successfully applied to different table understanding tasks, as in TAPAS (Herzig et al., 2020). A major limitation of these architectures is that they are constrained to classification-like tasks such as cell selection or entailment detection. We present TABT5, an encoder-decoder model that generates natural language text based on tables and textual inputs. TABT5 overcomes the encoder-only limitation by incorporating a decoder component and leverages the input structure with table specific embeddings and pre-training. TABT5 achieves new state-of-the-art results on several domains, including spreadsheet formula prediction with a 15% increase in sequence accuracy, QA with a 2.5% increase in sequence accuracy and data-to-text generation with a 2.5% increase in BLEU.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 20 of its 25 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

20 of 25 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  8. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  9. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  10. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  11. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  12. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
8 more
  1. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  2. 2021TABBIE: Pretrained Representations of Tabular Datacited by 111
  3. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  4. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  5. 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 90
  6. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  7. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  8. 2019Answering Conversational Questions on Structured Data without Logical Formscited by 24

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 8; 2024: 6; 2025: 3.

20 papers.

Sort:

By area:

  1. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
  2. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
  3. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  4. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  5. 2023HYTREL: Hypergraph-enhanced Tabular Data Representation Learningcited by 35
  6. 2023MultiTabQA: Generating Tabular Answers for Multi-Table Question Answeringcited by 32
  7. 2024Unifying Structured Data as Graph for Data-to-Text Pre-Trainingcited by 13
  8. 2023DOMINO: A Dual-System for Multi-step Visual Language Reasoningcited by 12
  9. 2023Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Taskscited by 10
  10. 2024Structsum Generation for Faster Text Comprehensioncited by 9
  11. 2024Generating Realistic Tabular Data with Large Language Modelscited by 9
  12. 2022APOLLO: An Optimized Training Approach for Long-form Numerical Reasoningcited by 8
8 more
  1. 2024Organizing Unstructured Image Collections using Natural Languagecited by 7
  2. 2024Faithful Chart Summarization with ChaTS-Picited by 5
  3. 2023Retrieval-Augmented Chain-of-Thought in Semi-structured Domainscited by 4
  4. 2022Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurationscited by 3
  5. 2023ReTAG: Reasoning Aware Table to Analytic Text Generationcited by 3
  6. 2025BenchPress: A Human-in-the-Loop Annotation System for Rapid Text-to-SQL Benchmark Curationcited by 3
  7. 2023Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learningcited by 2
  8. 2025FreshTab: Sourcing Fresh Data for Table-to-Text Generation Evaluationcited by 1