Data to 7 Oct 2026

Compositional generalization in a deep seq2seq model by separating syntax and semantics

arXiv 1904.09708 ↗First posted 22 Apr 2019cs.LG cs.CL stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in neuroscience suggesting separate brain systems for syntactic and semantic processing, we implement a modification to standard approaches in neural machine translation, imposing an analogous separation. The novel model, which we call Syntactic Attention, substantially outperforms standard methods in deep learning on the SCAN dataset, a compositional generalization task, without any hand-engineered features or additional supervision. Our work suggests that separating syntactic from semantic learning may be a useful heuristic for capturing compositional structure.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 17; 2023: 12; 2024: 5; 2025: 2; 2026: 2.

56 papers.

Sort:

By area:

2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
2022Compositional Semantic Parsing with Large Language Modelscited by 95
2021Making Transformers Solve Compositional Taskscited by 72
2021Meta-Learning to Compositionally Generalizecited by 62
2021The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalizationcited by 62
2023Compositional Generalization from First Principlescited by 62
2022Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learningcited by 59
2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46
2021ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parsercited by 35

38 more

2021On Compositional Generalization of Neural Machine Translationcited by 34
2021Learning Algebraic Recombination for Compositional Generalizationcited by 31
2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28
2023Sparse Universal Transformercited by 28
2023The Relational Bottleneck as an Inductive Bias for Efficient Abstractioncited by 26
2022On the Compositional Generalization Gap of In-Context Learningcited by 24
2022Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Modelscited by 22
2022Revisiting the Compositional Generalization Abilities of Neural Sequence Modelscited by 20
2021Structured Reordering for Modeling Latent Alignments in Sequence Transductioncited by 17
2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17
2021Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCANcited by 14
2022Improving Systematic Generalization Through Modularity and Augmentationcited by 13
2023Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Contextcited by 13
2021Improving Compositional Generalization in Classification Tasks via Structure Annotationscited by 12
2022Neural-Symbolic Recursive Machine for Systematic Generalizationcited by 12
2022Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Modelscited by 10
2023Compositional Generalization for Multi-label Text Classification: A Data-Augmentation Approachcited by 7
2021A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semanticscited by 6
2021Grounded Graph Decoding Improves Compositional Generalization in Question Answeringcited by 6
2021Learning to Generalize Compositionally by Transferring Across Semantic Parsing Taskscited by 6
2022Compositional Generalization and Decomposition in Neural Program Synthesiscited by 6
2023Visual Referential Games Further the Emergence of Disentangled Representationscited by 5
2024Towards Understanding the Relationship between In-context Learning and Compositional Generalizationcited by 5
2024What makes Models Compositional? A Theoretical View: With Supplementcited by 5
2024Compositional Generalization Across Distributional Shifts with Sparse Tree Operationscited by 5
2022Equivariant Transduction through Invariant Alignmentcited by 3
2023Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalizationcited by 3
2022Meta-Referential Games to Learn Compositional Learning Behaviourscited by 2
2023Layer-wise Representation Fusion for Compositional Generalizationcited by 2
2023Towards Equipping Transformer with the Ability of Systematic Compositionalitycited by 2
2023Pretrained Embeddings for E-commerce Machine Learning: When it Fails and Why?cited by 1
2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
2025Cognitive Decision Routing in Large Language Models: When to Think Fast, When to Think Slowcited by 1
2021Concepts, Properties and an Approach for Compositional Generalizationcited by 0
2022Is the Computation of Abstract Sameness Relations Human-Like in Neural Language Models?cited by 0
2022On a Built-in Conflict between Deep Learning and Systematic Generalizationcited by 0
2022Spontaneous Emerging Preference in Two-tower Language Modelcited by 0

50 most cited of 56 All 56 (JSON)