# Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks

arXiv: https://arxiv.org/abs/1711.00350 · HTML: https://mlsanity.com/paper/1711.00350/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 19 of 19)

- [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022)
- [The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning](https://mlsanity.com/paper/2504.05081/) (2025)
- [Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics](https://mlsanity.com/paper/2109.13986/) (2021)
- [Leveraging the Inductive Bias of Large Language Models for Abstract Textual Reasoning](https://mlsanity.com/paper/2110.02370/) (2021)
- [Distinguishing rule- and exemplar-based generalization in learning systems](https://mlsanity.com/paper/2110.04328/) (2021)
- [Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models](https://mlsanity.com/paper/2210.03575/) (2022)
- [A Survey of Inductive Reasoning for Large Language Models](https://mlsanity.com/paper/2510.10182/) (2025)
- [Measuring Combinatorial Creativity in Large Language Models and Humans](https://mlsanity.com/paper/2509.21043/) (2025)
- [Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learning](https://mlsanity.com/paper/2204.01437/) (2022)
- [Exploring System 1 and 2 communication for latent reasoning in LLMs](https://mlsanity.com/paper/2510.00494/) (2025)
- [Meta-Reinforcement Learning with Self-Modifying Networks](https://mlsanity.com/paper/2202.02363/) (2022)
- [Updater-Extractor Architecture for Inductive World State Representations](https://mlsanity.com/paper/2104.05500/) (2021)
- [Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing](https://mlsanity.com/paper/2210.04878/) (2022)
- [Attribute Diversity Determines the Systematicity Gap in VQA](https://mlsanity.com/paper/2311.08695/) (2023)
- [Harnessing Dataset Cartography for Improved Compositional Generalization in Transformers](https://mlsanity.com/paper/2310.12118/) (2023)
- [Attention as Binding: A Vector-Symbolic Perspective on Transformer Reasoning](https://mlsanity.com/paper/2512.14709/) (2025)
- [Sequential Compositional Generalization in Multimodal Models](https://mlsanity.com/paper/2404.12013/) (2024)
- [Understanding and Enhancing the Use of Context for Machine Translation](https://mlsanity.com/paper/2102.10437/) (2021)
- [SoftModel: A Neural Model That Grows Its Own Topology -- Governed Structural Growth for Continual In-Service Learning](https://mlsanity.com/paper/2608.16409/) (2026)

## Abstract (arXiv)

Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb "dax," he or she can immediately understand the meaning of "dax twice" or "sing and dax." In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply "mix-and-match" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the "dax" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.