mlsanityData to 7 Oct 2026

TableFormer: Robust Transformer Modeling for Table-Text Encoding

arXiv 2203.00274 ↗First posted 1 Mar 2022cs.CLcited by 90

Not on the map. Why?

Abstract (arXiv)

Understanding tables is an important aspect of natural language understanding. Existing models for table understanding require linearization of the table structure, where row or column order is encoded as an unwanted bias. Such spurious biases make the model vulnerable to row and column order perturbations. Additionally, prior work has not thoroughly modeled the table structures or table-text alignments, hindering the table-text understanding ability. In this work, we propose a robust and structurally aware table-text encoding architecture TableFormer, where tabular structural biases are incorporated completely through learnable attention biases. TableFormer is (1) strictly invariant to row and column orders, and, (2) could understand tables better due to its tabular inductive biases. Our evaluations showed that TableFormer outperforms strong baselines in all settings on SQA, WTQ and TabFact table reasoning datasets, and achieves state-of-the-art performance on SQA, especially when facing answer-invariant row and column order perturbations (6% improvement over the best baseline), because previous SOTA models' performance drops by 4% - 6% when facing such perturbations while TableFormer is not affected.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 1 Mar 2022.

Builds on

17 of 21 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  5. 2018Self-Attention with Relative Position Representationscited by 1,112
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  7. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  8. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  9. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  10. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  11. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  12. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
5 more
  1. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  2. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  3. 2020Understanding tables with intermediate pre-trainingcited by 68
  4. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  5. 2019Answering Conversational Questions on Structured Data without Logical Formscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 18; 2023: 20; 2024: 26; 2025: 14; 2026: 12.

90 papers.

Sort:

By area:

  1. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  2. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  3. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 177
  4. 2022TransTab: Learning Transferable Tabular Transformers Across Tablescited by 132
  5. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 120
  6. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  7. 2024TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarioscited by 76
  8. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 69
  9. 2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 51
  10. 2022A Survey on Table Question Answering: Recent Advancescited by 41
  11. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 41
  12. 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
38 more
  1. 2023HYTREL: Hypergraph-enhanced Tabular Data Representation Learningcited by 35
  2. 2024TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Datacited by 32
  3. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
  4. 2023Large Language Models are Complex Table Parserscited by 28
  5. 2023Active Prompt Learning in Vision Language Modelscited by 25
  6. 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
  7. 2024ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Contextcited by 23
  8. 2022Table-To-Text generation and pre-training with TabT5cited by 20
  9. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  10. 2022Tele-Knowledge Pre-training for Fault Analysiscited by 18
  11. 2022SUBS: Subtree Substitution for Compositional Semantic Parsingcited by 16
  12. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 15
  13. 2024FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answeringcited by 14
  14. 2023NameGuess: Column Name Expansion for Tabular Datacited by 13
  15. 2024Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretrainingcited by 12
  16. 2022Table Retrieval May Not Necessitate Table-specific Model Designcited by 11
  17. 2022Designing Robust Transformers using Robust Kernel Density Estimationcited by 11
  18. 2023Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Taskscited by 10
  19. 2024cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Paperscited by 10
  20. 2024Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysiscited by 9
  21. 2023cTBLS: Augmenting Large Language Models with Conversational Tablescited by 7
  22. 2023Faithful Low-Resource Data-to-Text Generation through Cycle Trainingcited by 7
  23. 2024UniDM: A Unified Framework for Data Manipulation with Large Language Modelscited by 7
  24. 2024Elliptical Attentioncited by 7
  25. 2025Table as a Modality for Large Language Modelscited by 7
  26. 2024Synthetic SQL Column Descriptions and Their Impact on Text-to-SQL Performancecited by 6
  27. 2023Bridge the Gap between Language models and Tabular Understandingcited by 5
  28. 2023Transformers as Graph-to-Graph Modelscited by 5
  29. 2023Think While You Write: Hypothesis Verification Promotes Faithful Knowledge-to-Text Generationcited by 5
  30. 2024ProTransformer: Robustify Transformers via Plug-and-Play Paradigmcited by 5
  31. 2025Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Modelscited by 5
  32. 2022A Numerical Reasoning Question Answering System with Fine-grained Retriever and the Ensemble of Multiple Generators for FinQAcited by 4
  33. 2022Realistic Data Augmentation Framework for Enhancing Tabular Reasoningcited by 4
  34. 2022NAPG: Non-Autoregressive Program Generation for Hybrid Tabular-Textual Question Answeringcited by 4
  35. 2023Diable: Efficient Dialogue State Tracking as Operations on Tablescited by 4
  36. 2023Scaling Experiments in Self-Supervised Cross-Table Representation Learningcited by 4
  37. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 4
  38. 2022FormLM: Recommending Creation Ideas for Online Forms by Modelling Semantic and Structural Informationcited by 3

50 most cited of 90 All 90 (JSON)