# Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment arXiv: https://arxiv.org/abs/2205.02054 ยท HTML: https://mlsanity.com/paper/2205.02054/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 38 references that are in the map. - First posted on arXiv 4 May 2022. ## Builds on (most cited first, 23 of 23) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019) - [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018) - [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019) - [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019) - [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020) - [GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2009.13845/) (2020) - [Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?](https://mlsanity.com/paper/2010.12725/) (2020) - [Improving Text-to-SQL Evaluation Methodology](https://mlsanity.com/paper/1806.09029/) (2018) - [Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures](https://mlsanity.com/paper/2007.08970/) (2020) - [Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization](https://mlsanity.com/paper/2109.05157/) (2021) - [Compositional Generalization via Neural-Symbolic Stack Machines](https://mlsanity.com/paper/2008.06662/) (2020) - [Span-based Semantic Parsing for Compositional Generalization](https://mlsanity.com/paper/2009.06040/) (2020) - [SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task](https://mlsanity.com/paper/1810.05237/) (2018) - [Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing](https://mlsanity.com/paper/1905.06241/) (2019) - [Natural SQL: Making SQL Easier to Infer from Natural Language Specifications](https://mlsanity.com/paper/2109.05153/) (2021) - [Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training](https://mlsanity.com/paper/2012.10309/) (2020) - [Meta-Learning to Compositionally Generalize](https://mlsanity.com/paper/2106.04252/) (2021) - [Compositional Generalization by Learning Analytical Expressions](https://mlsanity.com/paper/2006.10627/) (2020) - [Improving Compositional Generalization in Semantic Parsing](https://mlsanity.com/paper/2010.05647/) (2020) - [Structured Reordering for Modeling Latent Alignments in Sequence Transduction](https://mlsanity.com/paper/2106.03257/) (2021) - [SpatialNLI: A Spatial Domain Natural Language Interface to Databases Using Spatial Comprehension](https://mlsanity.com/paper/1908.10917/) (2019) - [Transferable Natural Language Interface to Structured Queries aided by Adversarial Generation](https://mlsanity.com/paper/1812.01245/) (2018) ## Built on by (most cited first, 16 of 16) - [Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL](https://mlsanity.com/paper/2406.08426/) (2024) - [A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability](https://mlsanity.com/paper/2303.13547/) (2023) - [Compositional Semantic Parsing with Large Language Models](https://mlsanity.com/paper/2209.15003/) (2022) - [A Survey on Employing Large Language Models for Text-to-SQL Tasks](https://mlsanity.com/paper/2407.15186/) (2024) - [Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey](https://mlsanity.com/paper/2310.17894/) (2023) - [Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques](https://mlsanity.com/paper/2305.17378/) (2023) - [Divide and Prompt: Chain of Thought Prompting for Text-to-SQL](https://mlsanity.com/paper/2304.11556/) (2023) - [Semantic Decomposition of Question and SQL for Text-to-SQL Parsing](https://mlsanity.com/paper/2310.13575/) (2023) - [Adapt and Decompose: Efficient Generalization of Text-to-SQL via Domain Adapted Least-To-Most Prompting](https://mlsanity.com/paper/2308.02582/) (2023) - [The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models](https://mlsanity.com/paper/2310.02777/) (2023) - [EHR-SeqSQL : A Sequential Text-to-SQL Dataset For Interactively Exploring Electronic Health Records](https://mlsanity.com/paper/2406.00019/) (2024) - [Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities](https://mlsanity.com/paper/2505.23838/) (2025) - [Exploring the Compositional Generalization in Context Dependent Text-to-SQL Parsing](https://mlsanity.com/paper/2306.04480/) (2023) - [Semantic Parsing for Complex Data Retrieval: Targeting Query Plans vs. SQL for No-Code Access to Relational Databases](https://mlsanity.com/paper/2312.14798/) (2023) - [Beyond Text-to-SQL for IoT Defense: A Comprehensive Framework for Querying and Classifying IoT Threats](https://mlsanity.com/paper/2406.17574/) (2024) - [PExA: Parallel Exploration Agent for Complex Text-to-SQL](https://mlsanity.com/paper/2604.22934/) (2026) ## Abstract (arXiv) In text-to-SQL tasks -- as in much of NLP -- compositional generalization is a major challenge: neural networks struggle with compositional generalization where training and test distributions differ. However, most recent attempts to improve this are based on word-level synthetic data or specific dataset splits to generate compositional biases. In this work, we propose a clause-level compositional example generation method. We first split the sentences in the Spider text-to-SQL dataset into sub-sentences, annotating each sub-sentence with its corresponding SQL clause, resulting in a new dataset Spider-SS. We then construct a further dataset, Spider-CG, by composing Spider-SS sub-sentences in different combinations, to test the ability of models to generalize compositionally. Experiments show that existing models suffer significant performance degradation when evaluated on Spider-CG, even though every sub-sentence is seen during training. To deal with this problem, we modify a number of state-of-the-art models to train on the segmented data of Spider-SS, and we show that this method improves the generalization performance.