Data to 7 Oct 2026

Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization

arXiv 2109.05157 ↗First posted 11 Sep 2021cs.CLcited by 92

Not on the map. Why?

Abstract (arXiv)

Recently, there has been significant progress in studying neural networks for translating text descriptions into SQL queries under the zero-shot cross-domain setting. Despite achieving good performance on some public benchmarks, we observe that existing text-to-SQL models do not generalize when facing domain knowledge that does not frequently appear in the training data, which may render the worse prediction performance for unseen domains. In this work, we investigate the robustness of text-to-SQL models when the questions require rarely observed domain knowledge. In particular, we define five types of domain knowledge and introduce Spider-DK (DK is the abbreviation of domain knowledge), a human-curated dataset based on the Spider benchmark for text-to-SQL translation. NL questions in Spider-DK are selected from Spider, and we modify some samples by adding domain knowledge that reflects real-world question paraphrases. We demonstrate that the prediction accuracy dramatically drops on samples that require such domain knowledge, even if the domain knowledge appears in the training set, and the model provides the correct predictions for related training samples.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 11 Sep 2021.

Builds on

16 of 22 references.

References · 16

2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
2017Learning a Neural Semantic Parser from User Feedbackcited by 98
2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
2019Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsingcited by 74

4 more

2020Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Trainingcited by 65
2020Improving Compositional Generalization in Semantic Parsingcited by 42
2021Improving Text-to-SQL with Schema Dependency Learningcited by 25
2021GP: Context-free Grammar Pre-training for Text-to-SQL Parserscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 17; 2024: 22; 2025: 27; 2026: 18.

92 papers.

Sort:

By area:

2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
2025Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQLcited by 59
2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54

38 more

2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 53
2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 51
2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 49
2022Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expectcited by 47
2023Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategiescited by 44
2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
2024Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLMcited by 35
2024PURPLE: Making a Large Language Model a Better SQL Writercited by 34
2024Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmarkcited by 33
2023Evaluating Cross-Domain Text-to-SQL Models and Benchmarkscited by 32
2024SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Modelscited by 32
2022Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsingcited by 30
2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 25
2022Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graphcited by 22
2025DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AIcited by 22
2023Divide and Prompt: Chain of Thought Prompting for Text-to-SQLcited by 20
2023Interactive Text-to-SQL Generation via Editable Step-by-Step Explanationscited by 19
2022Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignmentcited by 16
2023Prompting GPT-3.5 for Text-to-SQL with De-semanticization and Skeleton Retrievalcited by 16
2024Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruningcited by 16
2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Mergingcited by 16
2024From Natural Language to SQL: Review of LLM-based Text-to-SQL Systemscited by 15
2025SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQLcited by 15
2023UNITE: A Unified Benchmark for Text-to-SQL Evaluationcited by 14
2025Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQLcited by 14
2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 14
2024ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQLcited by 13
2023Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chaincited by 12
2025Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agentscited by 11
2025Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQLcited by 11
2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
2025ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQLcited by 8
2022SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parserscited by 7
2024Grounding Natural Language to SQL Translation with Data-Based Self-Explanationscited by 7
2026SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQLcited by 7
2025Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotationcited by 6

50 most cited of 92 All 92 (JSON)