mlsanityData to 14 Oct 2026

Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation

arXiv 2212.09994 ↗First posted 20 Dec 2022cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

The robustness of Text-to-SQL parsers against adversarial perturbations plays a crucial role in delivering highly reliable applications. Previous studies along this line primarily focused on perturbations in the natural language question side, neglecting the variability of tables. Motivated by this, we propose the Adversarial Table Perturbation (ATP) as a new attacking paradigm to measure the robustness of Text-to-SQL models. Following this proposition, we curate ADVETA, the first robustness evaluation benchmark featuring natural and realistic ATPs. All tested state-of-the-art models experience dramatic performance drops on ADVETA, revealing models' vulnerability in real-world practices. To defend against ATP, we build a systematic adversarial training example generation framework tailored for better contextualization of tabular data. Experiments show that our approach not only brings the best robustness improvement against table-side perturbations but also substantially empowers models against NL-side perturbations. We release our benchmark and code at: https://github.com/microsoft/ContextualSP.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 41 of its 62 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

41 of 62 references.

References · 41

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  7. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  8. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  9. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  10. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  11. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  12. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
29 more
  1. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  2. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  3. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  4. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  5. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 308
  6. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  7. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 286
  8. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
  9. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  10. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 152
  11. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
  12. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 140
  13. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  14. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  15. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  16. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  17. 2021Open Domain Question Answering over Tables via Dense Retrievalcited by 74
  18. 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 60
  19. 2021Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Datacited by 45
  20. 2019Imperceptible Adversarial Attacks on Tabular Datacited by 35
  21. 2019A Comparative Survey of Recent Natural Language Interfaces for Databasescited by 33
  22. 2020IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generationcited by 29
  23. 2021Awakening Latent Grounding from Pretrained Language Models for Semantic Parsingcited by 28
  24. 2020Photon: A Robust Cross-Domain Text-to-SQL Systemcited by 26
  25. 2020DuoRAT: Towards Simpler Text-to-SQL Modelscited by 17
  26. 2019Content Enhanced BERT-based Text-to-SQL Generationcited by 16
  27. 2021On Robustness of Neural Semantic Parserscited by 14
  28. 2020MT-Teql: Evaluating and Augmenting Consistency of Text-to-SQL Models with Metamorphic Testingcited by 3
  29. 2014Embedding Word Similarity with Neural Machine Translationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 8; 2025: 5; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 120
  2. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 119
  3. 2023SCM: Enhancing Large Language Model with Self-Controlled Memory Frameworkcited by 98
  4. 2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
  5. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 53
  6. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 43
  7. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  8. 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
  9. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  10. 2024TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoringcited by 17
  11. 2024From Natural Language to SQL: Review of LLM-based Text-to-SQL Systemscited by 16
  12. 2023Error Detection for Text-to-SQL Semantic Parsingcited by 12
16 more
  1. 2024Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQLcited by 11
  2. 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
  3. 2024SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQAcited by 9
  4. 2025LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Researchcited by 6
  5. 2026EvoSchema: Towards Text-to-SQL Robustness Against Schema Evolutioncited by 5
  6. 2024Improving Generalization in Semantic Parsing by Increasing Natural Language Variationcited by 4
  7. 2025TinySQL: A Progressive Text-to-SQL Dataset for Mechanistic Interpretability Researchcited by 2
  8. 2023On the Structural Generalization in Text-to-SQLcited by 1
  9. 2025Squrve: A Unified and Modular Framework for Complex Real-World Text-to-SQL Taskscited by 1
  10. 2025SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gapscited by 1
  11. 2026SQLStructEval: Structural Evaluation of LLM Text-to-SQL Generationcited by 1
  12. 2023Comprehensive Solution Program Centric Pretraining for Table-and-Text Hybrid Numerical Reasoningcited by 0
  13. 2026Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQLcited by 0
  14. 2026ESQ-Bench: A Multi-Tier Enterprise Oracle Benchmark for Evaluating NL2SQL Dialect Generalization and Silent Semantic Divergencecited by 0
  15. 2026TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysiscited by 0
  16. 2026DRL: A Deterministic Relational Middleware Layer for Transaction-Safe Enterprise NL2SQL Under Schema-Graph Scalingcited by 0