mlsanityData to 7 Oct 2026

Improving Compositional Generalization in Classification Tasks via Structure Annotations

arXiv 2106.10434 ↗First posted 19 Jun 2021cs.LG cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. Although humans seem to have a great ability to generalize compositionally, state-of-the-art neural models struggle to do so. In this work, we study compositional generalization in classification tasks and present two main contributions. First, we study ways to convert a natural language sequence-to-sequence dataset to a classification dataset that also requires compositional generalization. Second, we show that providing structural hints (specifically, providing parse trees and entity links as attention masks for a Transformer model) helps compositional generalization.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 10 of its 17 references that are in the map. First posted on arXiv 19 Jun 2021.

Builds on

10 of 17 references.

References · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2018Self-Attention with Relative Position Representationscited by 1,112
  4. 2018Universal Transformerscited by 613
  5. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  6. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  7. 2019Good-Enough Compositional Data Augmentationcited by 158
  8. 2020Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architecturescited by 95
  9. 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
  10. 2018Memorize or generalize? Searching for a compositional RNN in a haystackcited by 41

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 5.

12 papers.

  1. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  2. 2021Making Transformers Solve Compositional Taskscited by 72
  3. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 34
  4. 2022Structural generalization is hard for sequence-to-sequence modelscited by 24
  5. 2023Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Groundingcited by 17
  6. 2021Iterative Decoding for Compositional Generalization in Transformerscited by 6
  7. 2022Breakpoint Transformers for Modeling and Tracking Intermediate Beliefscited by 6
  8. 2022Compositional Generalization Requires Compositional Parserscited by 2
  9. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  10. 2023Joint Dropout: Improving Generalizability in Low-Resource Neural Machine Translation through Phrase Pair Variablescited by 1
  11. 2023Syntax-Guided Transformers: Elevating Compositional Generalization and Grounding in Multimodal Environmentscited by 1
  12. 2023Predicting generalization performance with correctness discriminatorscited by 1