mlsanityData to 7 Oct 2026

Compositional Generalisation with Structured Reordering and Fertility Layers

arXiv 2210.03183 ↗First posted 6 Oct 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Seq2seq models have been shown to struggle with compositional generalisation, i.e. generalising to new and potentially more complex structures than seen during training. Taking inspiration from grammar-based models that excel at compositional generalisation, we present a flexible end-to-end differentiable neural model that composes two structural operations: a fertility step, which we introduce in this work, and a reordering step based on previous work (Wang et al., 2021). To ensure differentiability, we use the expected value of each step. Our model outperforms seq2seq models by a wide margin on challenging compositional splits of realistic semantic parsing tasks that require generalisation to longer examples. It also compares favourably to other models targeting compositional generalisation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

30 of 49 references.

References · 30

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  3. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
  4. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 217
  5. 2019Good-Enough Compositional Data Augmentationcited by 158
  6. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  7. 2018Improving Text-to-SQL Evaluation Methodologycited by 133
  8. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  9. 2016Modeling Coverage for Neural Machine Translationcited by 110
  10. 2017Structured Attention Networkscited by 95
  11. 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
  12. 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
18 more
  1. 2021Making Transformers Solve Compositional Taskscited by 72
  2. 2021Meta-Learning to Compositionally Generalizecited by 62
  3. 2017A Minimal Span-Based Neural Constituency Parsercited by 59
  4. 2020The EOS Decision and Length Extrapolationcited by 46
  5. 2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46
  6. 2020Improving Compositional Generalization in Semantic Parsingcited by 42
  7. 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
  8. 2021Learning Algebraic Recombination for Compositional Generalizationcited by 31
  9. 2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
  10. 2021Disentangled Sequence to Sequence Learning for Compositional Generalizationcited by 30
  11. 2022Unobserved Local Structures Make Compositional Generalization Hardcited by 29
  12. 2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28
  13. 2019Exact Hard Monotonic Attention for Character-Level Transductioncited by 22
  14. 2021Structured Reordering for Modeling Latent Alignments in Sequence Transductioncited by 17
  15. 2016Incorporating Structural Alignment Biases into an Attentional Neural Translation Modelcited by 16
  16. 2016Online Segment to Segment Neural Transductioncited by 14
  17. 2018Sparse and Constrained Attention for Neural Machine Translationcited by 13
  18. 2021Compositional Generalization for Natural Language Interfaces to Web APIscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 0; 2026: 1.

6 papers.

  1. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  2. 2024Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generationcited by 8
  3. 2023Compositional Generalization without Trees using Multiset Tagging and Latent Permutationscited by 5
  4. 2023Structural generalization in COGS: Supertagging is (almost) all you needcited by 4
  5. 2024Compositional Generalization with Grounded Language Modelscited by 1
  6. 2026Do Recipes Have Personas? Characterizing and Generating Creator Style in Attributed Procedural Graphscited by 0