mlsanityData to 14 Oct 2026

On Compositionality in Neural Machine Translation

arXiv 1911.01497 ↗First posted 4 Nov 2019cs.CL cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

We investigate two specific manifestations of compositionality in Neural Machine Translation (NMT) : (1) Productivity - the ability of the model to extend its predictions beyond the observed length in training data and (2) Systematicity - the ability of the model to systematically recombine known parts and rules. We evaluate a standard Sequence to Sequence model on tests designed to assess these two properties in NMT. We quantitatively demonstrate that inadequate temporal processing, in the form of poor encoder representations is a bottleneck for both Productivity and Systematicity. We propose a simple pre-training mechanism which alleviates model performance on the two properties and leads to a significant improvement in BLEU scores.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). First posted on arXiv 4 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 4; 2023: 4; 2024: 1.

14 papers.

Sort:

  1. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  2. 2021The paradox of the compositionality of natural language: a neural machine translation case studycited by 59
  3. 2021On Compositional Generalization of Neural Machine Translationcited by 34
  4. 2021How BPE Affects Memorization in Transformerscited by 29
  5. 2022Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translationcited by 28
  6. 2021Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematicscited by 27
  7. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  8. 2022Finding Memo: Extractive Memorization in Constrained Sequence Generation Taskscited by 12
  9. 2023The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarkscited by 12
  10. 2023Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalizationcited by 3
  11. 2022Categorizing Semantic Representations for Neural Machine Translationcited by 2
  12. 2023Layer-wise Representation Fusion for Compositional Generalizationcited by 2
2 more
  1. 2024Evaluating Structural Generalization in Neural Machine Translationcited by 2
  2. 2023On Using Distribution-Based Compositionality Assessment to Evaluate Compositional Generalisation in Machine Translationcited by 1