mlsanityData to 13 Oct 2026

Revisiting the Compositional Generalization Abilities of Neural Sequence Models

arXiv 2203.07402 ↗First posted 14 Mar 2022cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Compositional generalization is a fundamental trait in humans, allowing us to effortlessly combine known phrases to form novel sentences. Recent works have claimed that standard seq-to-seq models severely lack the ability to compositionally generalize. In this paper, we focus on one-shot primitive generalization as introduced by the popular SCAN benchmark. We demonstrate that modifying the training distribution in simple and intuitive ways enables standard seq-to-seq models to achieve near-perfect generalization performance, thereby showing that their compositional generalization abilities were previously underestimated. We perform detailed empirical analysis of this phenomenon. Our results indicate that the generalization performance of models is highly sensitive to the characteristics of the training data which should be carefully considered while designing such benchmarks in future.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

16 of 23 references.

References · 16

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  2. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 261
  3. 2019Good-Enough Compositional Data Augmentationcited by 158
  4. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  5. 2019Compositional generalization through meta sequence-to-sequence learningcited by 99
  6. 2020Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architecturescited by 95
  7. 2020Compositional Generalization via Neural-Symbolic Stack Machinescited by 80
  8. 2021Making Transformers Solve Compositional Taskscited by 72
  9. 2020Sequence-Level Mixed Sample Data Augmentationcited by 71
  10. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
  11. 2021Meta-Learning to Compositionally Generalizecited by 62
  12. 2019Linguistic generalization and compositionality in modern artificial neural networkscited by 58
4 more
  1. 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
  2. 2019Compositional Generalization for Primitive Substitutionscited by 48
  3. 2019Human few-shot learning of compositional instructionscited by 47
  4. 2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 9; 2024: 4; 2025: 2.

20 papers.

Sort:

By area:

  1. 2022Unveiling Transformers with LEGO: a synthetic reasoning taskcited by 106
  2. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  3. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  4. 2022Characterizing Intrinsic Compositionality in Transformers with Tree Projectionscited by 40
  5. 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 25
  6. 2023ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretationcited by 24
  7. 2023The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarkscited by 12
  8. 2022When Can Transformers Ground and Compose: Insights from Compositional Generalization Benchmarkscited by 11
  9. 2024Benchmarking GPT-4 on Algorithmic Problems: A Systematic Evaluation of Prompting Strategiescited by 9
  10. 2022Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionalitycited by 8
  11. 2023MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretationscited by 7
  12. 2023Data Factors for Better Compositional Generalizationcited by 5
8 more
  1. 2024Towards Understanding the Relationship between In-context Learning and Compositional Generalizationcited by 5
  2. 2024Compositional Generalization Across Distributional Shifts with Sparse Tree Operationscited by 5
  3. 2023Data Distribution Bottlenecks in Grounding Language Models to Knowledge Basescited by 3
  4. 2023Attribute Diversity Determines the Systematicity Gap in VQAcited by 3
  5. 2023Harnessing Dataset Cartography for Improved Compositional Generalization in Transformerscited by 2
  6. 2024LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translationcited by 1
  7. 2025Data Distributional Properties As Inductive Bias for Systematic Generalizationcited by 0
  8. 2025Compositionality in Time Series: A Proof of Concept using Symbolic Dynamics and Compositional Data Augmentationcited by 0