mlsanityData to 7 Oct 2026

Lexicon Learning for Few-Shot Neural Sequence Modeling

arXiv 2106.03993 ↗First posted 7 Jun 2021cs.CL cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Sequence-to-sequence transduction is the core problem in language processing applications as diverse as semantic parsing, machine translation, and instruction following. The neural network models that provide the dominant solution to these problems are brittle, especially in low-resource settings: they fail to generalize correctly or systematically from small datasets. Past work has shown that many failures of systematic generalization arise from neural models' inability to disentangle lexical phenomena from syntactic ones. To address this, we augment neural decoders with a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules. We describe how to initialize this mechanism using a variety of lexicon learning algorithms, and show that it improves systematic generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

21 of 42 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  4. 2017Convolutional Sequence to Sequence Learningcited by 550
  5. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  6. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
  7. 2018Deep Learning: A Critical Appraisalcited by 214
  8. 2019Good-Enough Compositional Data Augmentationcited by 158
  9. 2016Data Recombination for Neural Semantic Parsingcited by 111
  10. 2020Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architecturescited by 95
  11. 2020Learning Compositional Rules via Neural Program Synthesiscited by 74
  12. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
9 more
  1. 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
  2. 2015Learning to Transduce with Unbounded Memorycited by 54
  3. 2020Compositional Generalization by Learning Analytical Expressionscited by 53
  4. 2019Human few-shot learning of compositional instructionscited by 47
  5. 2016Incorporating Discrete Translation Lexicons into Neural Machine Translationcited by 33
  6. 2017Improving Lexical Choice in Neural Machine Translationcited by 17
  7. 2018Towards one-shot learning for rare-word translation with external expertscited by 10
  8. 2019Pointer-based Fusion of Bilingual Lexicons into Neural Machine Translationcited by 3
  9. 2020Compositional Neural Machine Translation by Removing the Lexicon from Syntaxcited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 14; 2023: 10; 2024: 2; 2025: 1.

28 papers.

Sort:

By area:

  1. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  2. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  3. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  4. 2022Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptationcited by 44
  5. 2021Disentangled Sequence to Sequence Learning for Compositional Generalizationcited by 30
  6. 2022Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Modelscited by 26
  7. 2022Structural generalization is hard for sequence-to-sequence modelscited by 24
  8. 2023ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretationcited by 24
  9. 2023SLOG: A Structural Generalization Benchmark for Semantic Parsingcited by 22
  10. 2022Revisiting the Compositional Generalization Abilities of Neural Sequence Modelscited by 20
  11. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  12. 2022Compositionality as Lexical Symmetrycited by 8
16 more
  1. 2022Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionalitycited by 8
  2. 2023Learning to Substitute Spans towards Improving Compositional Generalizationcited by 8
  3. 2023MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretationscited by 7
  4. 2022Compositional Generalisation with Structured Reordering and Fertility Layerscited by 6
  5. 2023Compositional Generalization without Trees using Multiset Tagging and Latent Permutationscited by 5
  6. 2022Equivariant Transduction through Invariant Alignmentcited by 3
  7. 2022Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learningcited by 3
  8. 2023Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalizationcited by 3
  9. 2022Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentationcited by 2
  10. 2022Compositional Generalization Requires Compositional Parserscited by 2
  11. 2023Layer-wise Representation Fusion for Compositional Generalizationcited by 2
  12. 2023Harnessing Dataset Cartography for Improved Compositional Generalization in Transformerscited by 2
  13. 2023Efficient Model-Agnostic Multi-Group Equivariant Networkscited by 1
  14. 2024Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddingscited by 1
  15. 2024Sequential Compositional Generalization in Multimodal Modelscited by 1
  16. 2025Learning to Substitute Components for Compositional Generalizationcited by 1