mlsanityData to 7 Oct 2026

Understanding tables with intermediate pre-training

arXiv 2010.00571 ↗First posted 1 Oct 2020cs.CL cs.AI cs.IRcited by 68

Not on the map. Why?

Abstract (arXiv)

Table entailment, the binary classification task of finding if a sentence is supported or refuted by the content of a table, requires parsing language and table structure as well as numerical and discrete reasoning. While there is extensive work on textual entailment, table entailment is less well studied. We adapt TAPAS (Herzig et al., 2020), a table-based BERT model, to recognize entailment. Motivated by the benefits of data augmentation, we create a balanced dataset of millions of automatically created training examples which are learned in an intermediate step prior to fine-tuning. This new data is not only useful for table entailment, but also for SQA (Iyyer et al., 2017), a sequential table QA task. To be able to use long examples as input of BERT models, we evaluate table pruning techniques as a pre-processing step to drastically improve the training and prediction efficiency at a moderate drop in accuracy. The different methods set the new state-of-the-art on the TabFact (Chen et al., 2020) and SQA datasets.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 18; 2023: 11; 2024: 13; 2025: 6; 2026: 2.

68 papers.

Sort:

By area:

  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  2. 2021A Survey on Automated Fact-Checkingcited by 376
  3. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  4. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  5. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  6. 2022Binding Language Models in Symbolic Languagescited by 224
  7. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
  8. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  9. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 132
  10. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 120
  11. 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
  12. 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 90
38 more
  1. 2021Open Domain Question Answering over Tables via Dense Retrievalcited by 74
  2. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  3. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  4. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  5. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  6. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  7. 2024PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrieverscited by 42
  8. 2022PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-trainingcited by 41
  9. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 41
  10. 2024TableRAG: Million-Token Table Understanding with Language Modelscited by 40
  11. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  12. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
  13. 2023TempTabQA: Temporal Question Answering for Semi-Structured Tablescited by 27
  14. 2021Incorporating External Knowledge to Enhance Tabular Reasoningcited by 23
  15. 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
  16. 2024Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understandingcited by 20
  17. 2021FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretrainingcited by 19
  18. 2022Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQAcited by 17
  19. 2022SUBS: Subtree Substitution for Compositional Semantic Parsingcited by 16
  20. 2022PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generationcited by 16
  21. 2021TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-trainingcited by 14
  22. 2023From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuningcited by 14
  23. 2022Table-based Fact Verification with Self-adaptive Mixture of Expertscited by 12
  24. 2021Multi-modal Retrieval of Tables and Texts Using Tri-encoder Modelscited by 11
  25. 2021Exploring Decomposition for Table-based Fact Verificationcited by 11
  26. 2023Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Taskscited by 10
  27. 2024cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Paperscited by 10
  28. 2021Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoningcited by 9
  29. 2021Table-based Fact Verification with Salience-aware Learningcited by 9
  30. 2024SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQAcited by 9
  31. 2021DoT: An efficient Double Transformer for NLP tasks with tablescited by 8
  32. 2024Enhancing Temporal Understanding in LLMs for Semi-structured Tablescited by 8
  33. 2022Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Frameworkcited by 6
  34. 2023HistAlign: Improving Context Dependency in Language Generation by Aligning with Historycited by 6
  35. 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Setscited by 6
  36. 2024Faithful Chart Summarization with ChaTS-Picited by 5
  37. 2025RePanda: Pandas-powered Tabular Verification and Reasoningcited by 5
  38. 2021Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verificationcited by 4

50 most cited of 68 All 68 (JSON)