mlsanityData to 7 Oct 2026

Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks

arXiv 1807.07545 ↗First posted 19 Jul 2018cs.CL cs.AI cs.LGcited by 51

Not on the map. Why?

Abstract (arXiv)

Systematic compositionality is the ability to recombine meaningful units with regular and predictable outcomes, and it's seen as key to humans' capacity for generalization in language. Recent work has studied systematic compositionality in modern seq2seq models using generalization to novel navigation instructions in a grounded environment as a probing tool, requiring models to quickly bootstrap the meaning of new words. We extend this framework here to settings where the model needs only to recombine well-trained functional words (such as "around" and "right") in novel contexts. Our findings confirm and strengthen the earlier ones: seq2seq models can be impressively good at generalizing to novel combinations of previously-seen input, but only when they receive extensive training on the specific pattern to be generalized (e.g., generalizing from many examples of "X around right" to "jump around right"), while failing when generalization requires novel application of compositional rules (e.g., inferring the meaning of "around right" from those of "right" and "around").

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 16; 2023: 12; 2024: 4; 2025: 5; 2026: 2.

51 papers.

Sort:

By area:

  1. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  2. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  3. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  4. 2023Compositional Generalization from First Principlescited by 62
  5. 2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
  6. 2021On Compositional Generalization of Neural Machine Translationcited by 34
  7. 2023Dissociating language and thought in large language modelscited by 27
  8. 2023Neural-Logic Human-Object Interaction Detectioncited by 25
  9. 2021Dynamic Inference with Neural Interpreterscited by 24
  10. 2022Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Modelscited by 22
  11. 2024On Provable Length and Compositional Generalizationcited by 22
  12. 2021(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performancecited by 20
38 more
  1. 2021Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Taskscited by 19
  2. 2021Structured Reordering for Modeling Latent Alignments in Sequence Transductioncited by 17
  3. 2022SUBS: Subtree Substitution for Compositional Semantic Parsingcited by 16
  4. 2021How Modular Should Neural Module Networks Be for Systematic Generalization?cited by 15
  5. 2021Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCANcited by 14
  6. 2022Improving Systematic Generalization Through Modularity and Augmentationcited by 13
  7. 2023Sequential Query Encoding For Complex Query Answering on Knowledge Graphscited by 13
  8. 2021The Role of Syntactic Planning in Compositional Image Captioningcited by 12
  9. 2023The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarkscited by 12
  10. 2022Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluationcited by 8
  11. 2022Defending Compositionality in Emergent Languagescited by 8
  12. 2022Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionalitycited by 8
  13. 2021Generalization in Multimodal Language Learning from Simulationcited by 6
  14. 2023Visual Referential Games Further the Emergence of Disentangled Representationscited by 5
  15. 2023Data Factors for Better Compositional Generalizationcited by 5
  16. 2023Compositional Program Generation for Few-Shot Systematic Generalizationcited by 4
  17. 2022Understanding Robust Generalization in Learning Regular Languagescited by 3
  18. 2022Learning Parameterized Task Structure for Generalization to Unseen Entitiescited by 3
  19. 2022Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsingcited by 3
  20. 2023Studying Generalization on Memory-Based Methods in Continual Learningcited by 3
  21. 2023Modularity in Deep Learning: A Surveycited by 3
  22. 2022Meta-Referential Games to Learn Compositional Learning Behaviourscited by 2
  23. 2023On graph-based reentrancy-free semantic parsingcited by 2
  24. 2024IWISDM: Assessing instruction following in multimodal models at scalecited by 2
  25. 2025Systematic Generalization in Language Models Scales with Information Entropycited by 2
  26. 2022Measuring Alignment Bias in Neural Seq2Seq Semantic Parserscited by 1
  27. 2022NeuroCERIL: Robotic Imitation Learning via Hierarchical Cause-Effect Reasoning in Programmable Attractor Neural Networkscited by 1
  28. 2023Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraintscited by 1
  29. 2024Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddingscited by 1
  30. 2025Propositional Logic for Probing Generalization in Neural Networkscited by 1
  31. 2025Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out of Distribution Generalizationcited by 1
  32. 2021Intuitive Physics Guided Exploration for Sample Efficient Sim2real Transfercited by 0
  33. 2022Syntactic Inductive Biases for Deep Learning Methodscited by 0
  34. 2022A Short Survey of Systematic Generalizationcited by 0
  35. 2024Hierarchical Structure Enhances the Convergence and Generalizability of Linear Molecular Representationcited by 0
  36. 2025Algorithm Development in Neural Networks: Insights from the Streaming Parity Taskcited by 0
  37. 2025Can Constructions "SCAN" Compositionality ?cited by 0
  38. 2026Generalization in LLM Problem Solving: The Case of the Shortest Pathcited by 0

50 most cited of 51 All 51 (JSON)