# Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysis

arXiv: https://arxiv.org/abs/2406.12719 · HTML: https://mlsanity.com/paper/2406.12719/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [Stabilizing Transformer Training by Preventing Attention Entropy Collapse](https://mlsanity.com/paper/2303.06296/) (2023)
- [TableFormer: Robust Transformer Modeling for Table-Text Encoding](https://mlsanity.com/paper/2203.00274/) (2022)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations](https://mlsanity.com/paper/2306.14321/) (2023)
- [Rethinking Tabular Data Understanding with Large Language Models](https://mlsanity.com/paper/2312.16702/) (2023)
- [Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning](https://mlsanity.com/paper/2205.03972/) (2022)
- [FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering](https://mlsanity.com/paper/2404.18585/) (2024)
- [Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?](https://mlsanity.com/paper/2311.09109/) (2023)

## Built on by (most cited first, 9 of 9)

- [FLEXTAF: Enhancing Table Reasoning with Flexible Tabular Formats](https://mlsanity.com/paper/2408.08841/) (2024)
- [Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections](https://mlsanity.com/paper/2504.18333/) (2025)
- [Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation](https://mlsanity.com/paper/2510.09671/) (2025)
- [Analyzing LLM Instruction Optimization for Tabular Fact Verification](https://mlsanity.com/paper/2602.17937/) (2026)
- [The Power of Order: Fooling LLMs with Adversarial Table Permutations](https://mlsanity.com/paper/2605.00445/) (2026)
- [An Empirical Investigation of Robustness in Large Language Models under Tabular Distortions](https://mlsanity.com/paper/2601.05009/) (2026)
- [A Closer Look into LLMs for Table Understanding](https://mlsanity.com/paper/2603.15402/) (2026)
- [How to Utilize Complementary Vision-Text Information for 2D Structure Understanding](https://mlsanity.com/paper/2603.16245/) (2026)
- [TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysis](https://mlsanity.com/paper/2608.24145/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs), already shown to ace various unstructured text comprehension tasks, have also remarkably been shown to tackle table (structured) comprehension tasks without specific training. Building on earlier studies of LLMs for tabular tasks, we probe how in-context learning (ICL), model scale, instruction tuning, and domain bias affect Tabular QA (TQA) robustness by testing LLMs, under diverse augmentations and perturbations, on diverse domains: Wikipedia-based WTQ, financial TAT-QA, and scientific SCITAB. Although instruction tuning and larger, newer LLMs deliver stronger, more robust TQA performance, data contamination and reliability issues, especially on WTQ, remain unresolved. Through an in-depth attention analysis, we reveal a strong correlation between perturbation-induced shifts in attention dispersion and the drops in performance, with sensitivity peaking in the model's middle layers. We highlight the need for improved interpretable methodologies to develop more reliable LLMs for table comprehension. Through an in-depth attention analysis, we reveal a strong correlation between perturbation-induced shifts in attention dispersion and performance drops, with sensitivity peaking in the model's middle layers. Based on these findings, we argue for the development of structure-aware self-attention mechanisms and domain-adaptive processing techniques to improve the transparency, generalization, and real-world reliability of LLMs on tabular data.