mlsanityData to 7 Oct 2026

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations

arXiv 2306.14321 ↗First posted 25 Jun 2023cs.CL cs.AIcited by 41

Not on the map. Why?

Abstract (arXiv)

Despite significant progress having been made in question answering on tabular data (Table QA), it's unclear whether, and to what extent existing Table QA models are robust to task-specific perturbations, e.g., replacing key question entities or shuffling table columns. To systematically study the robustness of Table QA models, we propose a benchmark called RobuT, which builds upon existing Table QA datasets (WTQ, WikiSQL-Weak, and SQA) and includes human-annotated adversarial perturbations in terms of table header, table content, and question. Our results indicate that both state-of-the-art Table QA models and large language models (e.g., GPT-3) with few-shot learning falter in these adversarial sets. We propose to address this problem by using large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models. Our data and code is publicly available at https://github.com/yilunzhao/RobuT.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 25 Jun 2023.

Builds on

42 of 60 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  11. 2018Robustness May Be at Odds with Accuracycited by 873
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
30 more
  1. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  2. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  3. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  4. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  5. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  6. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  7. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  8. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  9. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  10. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  11. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 166
  12. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  13. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  14. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generationcited by 117
  15. 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
  16. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  17. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  18. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  19. 2020Logical Natural Language Generation from Open-Domain Tablescited by 90
  20. 2022TableFormer: Robust Transformer Modeling for Table-Text Encodingcited by 90
  21. 2020INFOTABS: Inference on Tables as Semi-structured Datacited by 84
  22. 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
  23. 2020Understanding tables with intermediate pre-trainingcited by 68
  24. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
  25. 2022OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answeringcited by 29
  26. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
  27. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  28. 2020Photon: A Robust Cross-Domain Text-to-SQL Systemcited by 26
  29. 2018Adversarial TableQA: Attention Supervision for Question Answering on Tablescited by 10
  30. 2020Generating Semantically Valid Adversarial Questions for TableQAcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 15; 2025: 19; 2026: 4.

41 papers.

Sort:

By area:

  1. 2024What matters when building vision-language models?cited by 321
  2. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  3. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  4. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  5. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  6. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 86
  7. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  8. 2024Large Language Model for Table Processing: A Surveycited by 52
  9. 2025FineVision: Open Data Is All You Needcited by 51
  10. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  11. 2023Rethinking Tabular Data Understanding with Large Language Modelscited by 34
  12. 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
29 more
  1. 2024ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Modelscited by 21
  2. 2025Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligencecited by 20
  3. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 20
  4. 2024FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documentscited by 19
  5. 2023Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?cited by 16
  6. 2024FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answeringcited by 14
  7. 2023Observatory: Characterizing Embeddings of Relational Tablescited by 12
  8. 2024DOGR: Towards Versatile Visual Document Grounding and Referringcited by 10
  9. 2024Exploring the Robustness of Language Models for Tabular Question Answering via Attention Analysiscited by 9
  10. 2024SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQAcited by 9
  11. 2025LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sightcited by 8
  12. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  13. 2025Breaking Language Barriers in Visual Language Models via Multilingual Textual Regularizationcited by 4
  14. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 4
  15. 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
  16. 2025PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Trickscited by 3
  17. 2025M3TQA: Massively Multilingual Multitask Table Question Answeringcited by 3
  18. 2024Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variabilitycited by 2
  19. 2025RoToR: Towards More Reliable Responses for Order-Invariant Inputscited by 2
  20. 2025Structural Deep Encoding for Table Question Answeringcited by 2
  21. 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUscited by 2
  22. 2025Data Metabolism: An Efficient Data Design Schema For Vision Language Modelcited by 1
  23. 2025GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMscited by 1
  24. 2026Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Modelscited by 1
  25. 2024Towards Building a Robust Knowledge Intensive Question Answering Model with Large Language Modelscited by 0
  26. 2025Capybara-OMNI: An Efficient Paradigm for Building Omni-Modal Language Modelscited by 0
  27. 2025DEVAL: A Framework for Evaluating and Improving the Derivation Capability of Large Language Modelscited by 0
  28. 2026An Empirical Investigation of Robustness in Large Language Models under Tabular Distortionscited by 0
  29. 2026TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexitycited by 0