mlsanityData to 14 Oct 2026

Structural Deep Encoding for Table Question Answering

arXiv 2503.01457 ↗First posted 3 Mar 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Although Transformers-based architectures excel at processing textual information, their naive adaptation for tabular data often involves flattening the table structure. This simplification can lead to the loss of essential inter-dependencies between rows, columns, and cells, while also posing scalability challenges for large tables. To address these issues, prior works have explored special tokens, structured embeddings, and sparse attention patterns. In this paper, we conduct a comprehensive analysis of tabular encoding techniques, which highlights the crucial role of attention sparsity in preserving structural information of tables. We also introduce a set of novel sparse attention mask designs for tabular data, that not only enhance computational efficiency but also preserve structural integrity, leading to better overall performance.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

31 of 40 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
  7. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  8. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  9. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  10. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  11. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
  12. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
19 more
  1. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
  2. 2021FeTaQA: Free-form Table Question Answeringcited by 175
  3. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  4. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  5. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  6. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  7. 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 91
  8. 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 60
  9. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  10. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 53
  11. 2024Large Language Model for Table Processing: A Surveycited by 52
  12. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 43
  13. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  14. 2019Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programscited by 26
  15. 2023Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniquescited by 24
  16. 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
  17. 2021Representations for Question Answering from Documents with Tables and Textcited by 21
  18. 2024FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answeringcited by 16
  19. 2024Improving Generalization in Semantic Parsing by Increasing Natural Language Variationcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Granite Embedding R2 Modelscited by 8
  2. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 5