# Capturing Row and Column Semantics in Transformer Based Question Answering over Tables

arXiv: https://arxiv.org/abs/2104.08303 · HTML: https://mlsanity.com/paper/2104.08303/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Retrospective Reader for Machine Reading Comprehension](https://mlsanity.com/paper/2001.09694/) (2020)
- [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://mlsanity.com/paper/1909.04849/) (2019)
- [SG-Net: Syntax-Guided Machine Reading Comprehension](https://mlsanity.com/paper/1908.05147/) (2019)
- [Span Selection Pre-training for Question Answering](https://mlsanity.com/paper/1909.04120/) (2019)
- [A BERT Baseline for the Natural Questions](https://mlsanity.com/paper/1901.08634/) (2019)
- [Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs](https://mlsanity.com/paper/1909.04165/) (2019)
- [Ad Hoc Table Retrieval using Semantic Similarity](https://mlsanity.com/paper/1802.06159/) (2018)
- [TabMCQ: A Dataset of General Knowledge Tables and Multiple-choice Questions](https://mlsanity.com/paper/1602.03960/) (2016)
- [Frustratingly Easy Natural Question Answering](https://mlsanity.com/paper/1909.05286/) (2019)
- [Schemaless Queries over Document Tables with Dependencies](https://mlsanity.com/paper/1911.09356/) (2019)

## Built on by (most cited first, 30 of 31)

- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industry](https://mlsanity.com/paper/2106.12944/) (2021)
- [A Survey on Table Question Answering: Recent Advances](https://mlsanity.com/paper/2207.05270/) (2022)
- [TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition](https://mlsanity.com/paper/2404.10150/) (2024)
- [HYTREL: Hypergraph-enhanced Tabular Data Representation Learning](https://mlsanity.com/paper/2307.08623/) (2023)
- [Open Domain Question Answering with A Unified Knowledge Interface](https://mlsanity.com/paper/2110.08417/) (2021)
- [Large Language Models are Complex Table Parsers](https://mlsanity.com/paper/2312.11521/) (2023)
- [TempTabQA: Temporal Question Answering for Semi-Structured Tables](https://mlsanity.com/paper/2311.08002/) (2023)
- [DiSCoMaT: Distantly Supervised Composition Extraction from Tables in Materials Science Articles](https://mlsanity.com/paper/2207.01079/) (2022)
- [GFS: Graph-based Feature Synthesis for Prediction over Relational Databases](https://mlsanity.com/paper/2312.02037/) (2023)
- [FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering](https://mlsanity.com/paper/2404.18585/) (2024)
- [End-to-End Table Question Answering via Retrieval-Augmented Generation](https://mlsanity.com/paper/2203.16714/) (2022)
- [API-Assisted Code Generation for Question Answering on Varied Table Structures](https://mlsanity.com/paper/2310.14687/) (2023)
- [Multi-Row, Multi-Span Distant Supervision For Table+Text Question](https://mlsanity.com/paper/2112.07337/) (2021)
- [CLTR: An End-to-End, Transformer-Based System for Cell Level Table Retrieval and Table Question Answering](https://mlsanity.com/paper/2106.04441/) (2021)
- [Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning](https://mlsanity.com/paper/2108.00578/) (2021)
- [Enhancing Temporal Understanding in LLMs for Semi-structured Tables](https://mlsanity.com/paper/2407.16030/) (2024)
- [Topic Transferable Table Question Answering](https://mlsanity.com/paper/2109.07377/) (2021)
- [Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework](https://mlsanity.com/paper/2212.10097/) (2022)
- [Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables](https://mlsanity.com/paper/2309.11049/) (2023)
- [DataFrame QA: A Universal LLM Framework on DataFrame Question Answering Without Data Exposure](https://mlsanity.com/paper/2401.15463/) (2024)
- [TIE: Topological Information Enhanced Structural Reading Comprehension on Web Pages](https://mlsanity.com/paper/2205.06435/) (2022)
- [Realistic Data Augmentation Framework for Enhancing Tabular Reasoning](https://mlsanity.com/paper/2210.12795/) (2022)
- [Enhancing Tabular Reasoning with Pattern Exploiting Training](https://mlsanity.com/paper/2210.12259/) (2022)
- [Leveraging Data Recasting to Enhance Tabular Reasoning](https://mlsanity.com/paper/2211.12641/) (2022)
- [Retrieval-Based Transformer for Table Augmentation](https://mlsanity.com/paper/2306.11843/) (2023)
- [InfoSync: Information Synchronization across Multilingual Semi-structured Tables](https://mlsanity.com/paper/2307.03313/) (2023)
- [Sheet As Token: A Graph-Enhanced Representation for Multi-Sheet Spreadsheet Understanding](https://mlsanity.com/paper/2605.05811/) (2026)
- [TACR: A Table-alignment-based Cell-selection and Reasoning Model for Hybrid Question-Answering](https://mlsanity.com/paper/2305.14682/) (2023)
- [Reimagining Retrieval Augmented Language Models for Answering Queries](https://mlsanity.com/paper/2306.01061/) (2023)

## Abstract (arXiv)

Transformer based architectures are recently used for the task of answering questions over tables. In order to improve the accuracy on this task, specialized pre-training techniques have been developed and applied on millions of open-domain web tables. In this paper, we propose two novel approaches demonstrating that one can achieve superior performance on table QA task without even using any of these specialized pre-training techniques. The first model, called RCI interaction, leverages a transformer based architecture that independently classifies rows and columns to identify relevant cells. While this model yields extremely high accuracy at finding cell values on recent benchmarks, a second model we propose, called RCI representation, provides a significant efficiency advantage for online QA systems over tables by materializing embeddings for existing tables. Experiments on recent benchmarks prove that the proposed methods can effectively locate cell values on tables (up to ~98% Hit@1 accuracy on WikiSQL lookup questions). Also, the interaction model outperforms the state-of-the-art transformer based approaches, pre-trained on very large table corpora (TAPAS and TaBERT), achieving ~3.4% and ~18.86% additional precision improvement on the standard WikiSQL benchmark.