Data to 7 Oct 2026

Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures

arXiv 2007.08970 ↗First posted 17 Jul 2020cs.CL cs.LGcited by 95

Not on the map. Why?

Abstract (arXiv)

While mainstream machine learning methods are known to have limited ability to compositionally generalize, new architectures and techniques continue to be proposed to address this limitation. We investigate state-of-the-art techniques and architectures in order to assess their effectiveness in improving compositional generalization in semantic parsing tasks based on the SCAN and CFQ datasets. We show that masked language model (MLM) pre-training rivals SCAN-inspired architectures on primitive holdout splits. On a more complex compositional task, we show that pre-training leads to significant improvements in performance vs. comparable non-pre-trained models, whereas architectures proposed to encourage compositional generalization on SCAN or in the area of algorithm learning fail to lead to significant improvements. We establish a new state of the art on the CFQ compositional generalization benchmark using MLM pre-training together with an intermediate representation.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 24; 2023: 27; 2024: 10; 2025: 6; 2026: 2.

95 papers.

Sort:

By area:

2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 319
2023What Algorithms can Transformers Learn? A Study in Length Generalizationcited by 198
2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 136
2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
2022Compositional Semantic Parsing with Large Language Modelscited by 95
2021Making Transformers Solve Compositional Taskscited by 72
2021Meta-Learning to Compositionally Generalizecited by 62
2021The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalizationcited by 62
2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
2023How Do In-Context Examples Affect Compositional Generalization?cited by 51

38 more

2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46
2021Systematic Generalization with Edge Transformerscited by 38
2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
2022Modern Baselines for SPARQL Semantic Parsingcited by 34
2021Learning Algebraic Recombination for Compositional Generalizationcited by 31
2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
2021Disentangled Sequence to Sequence Learning for Compositional Generalizationcited by 30
2022Unobserved Local Structures Make Compositional Generalization Hardcited by 29
2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28
2023Sparse Universal Transformercited by 28
2021Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalizationcited by 27
2022Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Modelscited by 26
2023Neural-Logic Human-Object Interaction Detectioncited by 25
2021Symbolic Behaviour in Artificial Intelligencecited by 24
2022Structural generalization is hard for sequence-to-sequence modelscited by 24
2022On the Compositional Generalization Gap of In-Context Learningcited by 24
2023Mixture of Soft Prompts for Controllable Data Generationcited by 23
2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Webcited by 23
2022Few-shot Subgoal Planning with Language Modelscited by 22
2022Revisiting the Compositional Generalization Abilities of Neural Sequence Modelscited by 20
2021Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Taskscited by 19
2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17
2022Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignmentcited by 16
2024A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practicescited by 15
2022Improving Systematic Generalization Through Modularity and Augmentationcited by 13
2022GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representationcited by 13
2024CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Eracited by 13
2021Improving Compositional Generalization in Classification Tasks via Structure Annotationscited by 12
2022LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Modelscited by 12
2023ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Modelscited by 12
2024Attention as a Hypernetworkcited by 12
2021SQALER: Scaling Question Answering by Decoupling Multi-Hop and Logical Reasoningcited by 10
2022Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Modelscited by 10
2023Semantic Parsing for Conversational Question Answering over Knowledge Graphscited by 10
2023Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Trackingcited by 10
2022Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionalitycited by 8
2023Learning to Substitute Spans towards Improving Compositional Generalizationcited by 8

50 most cited of 95 All 95 (JSON)