mlsanityData to 7 Oct 2026

Improving Generalization in Semantic Parsing by Increasing Natural Language Variation

arXiv 2402.08666 ↗First posted 13 Feb 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Text-to-SQL semantic parsing has made significant progress in recent years, with various models demonstrating impressive performance on the challenging Spider benchmark. However, it has also been shown that these models often struggle to generalize even when faced with small perturbations of previously (accurately) parsed expressions. This is mainly due to the linguistic form of questions in Spider which are overly specific, unnatural, and display limited variation. In this work, we use data augmentation to enhance the robustness of text-to-SQL parsers against natural language variations. Existing approaches generate question reformulations either via models trained on Spider or only introduce local changes. In contrast, we leverage the capabilities of large language models to generate more realistic and diverse questions. Using only a few prompts, we achieve a two-fold increase in the number of questions in Spider. Training on this augmented dataset yields substantial improvements on a range of evaluation sets, including robustness benchmarks and out-of-domain data.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 13 Feb 2024.

Builds on

32 of 43 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  10. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  11. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  12. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
20 more
  1. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  2. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  3. 2018Improving Text-to-SQL Evaluation Methodologycited by 133
  4. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  5. 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 123
  6. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
  7. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  8. 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
  9. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
  10. 2020Grounded Adaptation for Zero-shot Executable Semantic Parsingcited by 76
  11. 2021Natural SQL: Making SQL Easier to Infer from Natural Language Specificationscited by 66
  12. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
  13. 2021Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Datacited by 43
  14. 2021Learning to Synthesize Data for Semantic Parsingcited by 41
  15. 2021Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsingcited by 32
  16. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  17. 2022Importance of Synthesizing High-quality Data for Text-to-SQL Parsingcited by 27
  18. 2021On Robustness of Neural Semantic Parserscited by 14
  19. 2019SpatialNLI: A Spatial Domain Natural Language Interface to Databases Using Spatial Comprehensioncited by 5
  20. 2020ColloQL: Robust Cross-Domain Text-to-SQL Over Search Queriescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsingcited by 12
  2. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  3. 2025Structural Deep Encoding for Table Question Answeringcited by 2
  4. 2026LLM NL2SQL Robustness: Surface Noise vs. Linguistic Variation in Traditional and Agentic Settingscited by 0