# CNNs found to jump around more skillfully than RNNs: Compositional generalization in seq2seq convolutional networks

arXiv: https://arxiv.org/abs/1905.08527 · HTML: https://mlsanity.com/paper/1905.08527/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2019.

## Built on by (most cited first, 22 of 22)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization](https://mlsanity.com/paper/2110.07732/) (2021)
- [On Compositional Generalization of Neural Machine Translation](https://mlsanity.com/paper/2105.14802/) (2021)
- [Sequence-to-Sequence Learning with Latent Neural Grammars](https://mlsanity.com/paper/2109.01135/) (2021)
- [Emergent Communication: Generalization and Overfitting in Lewis Games](https://mlsanity.com/paper/2209.15342/) (2022)
- [Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models](https://mlsanity.com/paper/2210.06475/) (2022)
- [Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks](https://mlsanity.com/paper/2109.15256/) (2021)
- [A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practices](https://mlsanity.com/paper/2406.08787/) (2024)
- [Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN](https://mlsanity.com/paper/2107.01366/) (2021)
- [Neural-Symbolic Recursive Machine for Systematic Generalization](https://mlsanity.com/paper/2210.01603/) (2022)
- [Defending Compositionality in Emergent Languages](https://mlsanity.com/paper/2206.04751/) (2022)
- [Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality](https://mlsanity.com/paper/2211.15578/) (2022)
- [From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks](https://mlsanity.com/paper/2405.15164/) (2024)
- [A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics](https://mlsanity.com/paper/2103.01403/) (2021)
- [Data Factors for Better Compositional Generalization](https://mlsanity.com/paper/2311.04420/) (2023)
- [Deep learning for laboratory earthquake prediction and autoregressive forecasting of fault zone stress](https://mlsanity.com/paper/2203.13313/) (2022)
- [Understanding Robust Generalization in Learning Regular Languages](https://mlsanity.com/paper/2202.09717/) (2022)
- [Layer-wise Representation Fusion for Compositional Generalization](https://mlsanity.com/paper/2307.10799/) (2023)
- [Systematic Generalization in Language Models Scales with Information Entropy](https://mlsanity.com/paper/2505.13089/) (2025)
- [Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings](https://mlsanity.com/paper/2402.06492/) (2024)
- [CompoST: A Benchmark for Analyzing the Ability of LLMs To Compositionally Interpret Questions in a QALD Setting](https://mlsanity.com/paper/2507.21257/) (2025)
- [Data Distributional Properties As Inductive Bias for Systematic Generalization](https://mlsanity.com/paper/2502.20499/) (2025)

## Abstract (arXiv)

Lake and Baroni (2018) introduced the SCAN dataset probing the ability of seq2seq models to capture compositional generalizations, such as inferring the meaning of "jump around" 0-shot from the component words. Recurrent networks (RNNs) were found to completely fail the most challenging generalization cases. We test here a convolutional network (CNN) on these tasks, reporting hugely improved performance with respect to RNNs. Despite the big improvement, the CNN has however not induced systematic rules, suggesting that the difference between compositional and non-compositional behaviour is not clear-cut.