# Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models

arXiv: https://arxiv.org/abs/2104.06483 · HTML: https://mlsanity.com/paper/2104.06483/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection](https://mlsanity.com/paper/1910.11493/) (2019)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [UniMorph 2.0: Universal Morphology](https://mlsanity.com/paper/1810.11101/) (2018)
- [The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection](https://mlsanity.com/paper/1810.07125/) (2018)
- [CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages](https://mlsanity.com/paper/1706.09031/) (2017)
- [SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection](https://mlsanity.com/paper/2006.11572/) (2020)
- [Pushing the Limits of Low-Resource Morphological Inflection](https://mlsanity.com/paper/1908.05838/) (2019)
- [Exact Hard Monotonic Attention for Character-Level Transduction](https://mlsanity.com/paper/1905.06319/) (2019)
- [Hard Non-Monotonic Attention for Character-Level Transduction](https://mlsanity.com/paper/1808.10024/) (2018)
- [Imitation Learning for Neural Morphological String Transduction](https://mlsanity.com/paper/1808.10701/) (2018)
- [Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models](https://mlsanity.com/paper/1705.06106/) (2017)
- [Align and Copy: UZH at SIGMORPHON 2017 Shared Task for Morphological Reinflection](https://mlsanity.com/paper/1707.01355/) (2017)

## Built on by (most cited first, 13 of 13)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Sparks of Science: Hypothesis Generation Using Structured Paper Data](https://mlsanity.com/paper/2504.12976/) (2025)
- [(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance](https://mlsanity.com/paper/2108.05682/) (2021)
- [Understanding Compositional Data Augmentation in Typologically Diverse Morphological Inflection](https://mlsanity.com/paper/2305.13658/) (2023)
- [An Investigation of Noise in Morphological Inflection](https://mlsanity.com/paper/2305.16581/) (2023)
- [Computational Morphology with Neural Network Approaches](https://mlsanity.com/paper/2105.09404/) (2021)
- [Character-aware Transformers Learn an Irregular Morphological Pattern Yet None Generalize Like Humans](https://mlsanity.com/paper/2602.14100/) (2026)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)
- [Is linguistically-motivated data augmentation worth it?](https://mlsanity.com/paper/2506.03593/) (2025)
- [OOVs in the Spotlight: How to Inflect them?](https://mlsanity.com/paper/2404.08974/) (2024)
- [Probing Subphonemes in Morphology Models](https://mlsanity.com/paper/2505.11297/) (2025)
- [Improving Low-Resource Morphological Inflection via Self-Supervised Objectives](https://mlsanity.com/paper/2506.05227/) (2025)
- [Transformers over-extend what humans underlearn: the case of Spanish L-shaped morphome](https://mlsanity.com/paper/2507.21556/) (2025)

## Abstract (arXiv)

Deep learning sequence models have been successfully applied to the task of morphological inflection. The results of the SIGMORPHON shared tasks in the past several years indicate that such models can perform well, but only if the training data cover a good amount of different lemmata, or if the lemmata that are inflected at test time have also been seen in training, as has indeed been largely the case in these tasks. Surprisingly, standard models such as the Transformer almost completely fail at generalizing inflection patterns when asked to inflect previously unseen lemmata -- i.e. under "wug test"-like circumstances. While established data augmentation techniques can be employed to alleviate this shortcoming by introducing a copying bias through hallucinating synthetic new word forms using the alphabet in the language at hand, we show that, to be more effective, the hallucination process needs to pay attention to substrings of syllable-like length rather than individual characters or stems. We report a significant performance improvement with our substring-based hallucination model over previous data hallucination methods when training and test data do not overlap in their lemmata.