# An Investigation of Noise in Morphological Inflection

arXiv: https://arxiv.org/abs/2305.16581 · HTML: https://mlsanity.com/paper/2305.16581/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [MTNT: A Testbed for Machine Translation of Noisy Text](https://mlsanity.com/paper/1809.00388/) (2018)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages](https://mlsanity.com/paper/1706.09031/) (2017)
- [SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection](https://mlsanity.com/paper/2006.11572/) (2020)
- [(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance](https://mlsanity.com/paper/2108.05682/) (2021)
- [Morphological Inflection Generation Using Character Sequence to Sequence Learning](https://mlsanity.com/paper/1512.06110/) (2015)
- [Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models](https://mlsanity.com/paper/2104.06483/) (2021)
- [Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection](https://mlsanity.com/paper/1606.00589/) (2016)
- [Imitation Learning for Neural Morphological String Transduction](https://mlsanity.com/paper/1808.10701/) (2018)
- [Unsupervised Morphological Paradigm Completion](https://mlsanity.com/paper/2005.00970/) (2020)
- [The Paradigm Discovery Problem](https://mlsanity.com/paper/2005.01630/) (2020)
- [Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models](https://mlsanity.com/paper/1705.06106/) (2017)
- [Align and Copy: UZH at SIGMORPHON 2017 Shared Task for Morphological Reinflection](https://mlsanity.com/paper/1707.01355/) (2017)
- [The SIGMORPHON 2020 Shared Task on Unsupervised Morphological Paradigm Completion](https://mlsanity.com/paper/2005.13756/) (2020)
- [Morphological Processing of Low-Resource Languages: Where We Are and What's Next](https://mlsanity.com/paper/2203.08909/) (2022)

## Built on by (most cited first, 3 of 3)

- [Probing Subphonemes in Morphology Models](https://mlsanity.com/paper/2505.11297/) (2025)
- [Improving Low-Resource Morphological Inflection via Self-Supervised Objectives](https://mlsanity.com/paper/2506.05227/) (2025)
- [GlossAssist -- A Tool to Simplify Corpus Creation and Study the Effect of NLP Models in Low-Resource Documentation Settings](https://mlsanity.com/paper/2606.04367/) (2026)

## Abstract (arXiv)

With a growing focus on morphological inflection systems for languages where high-quality data is scarce, training data noise is a serious but so far largely ignored concern. We aim at closing this gap by investigating the types of noise encountered within a pipeline for truly unsupervised morphological paradigm completion and its impact on morphological inflection systems: First, we propose an error taxonomy and annotation pipeline for inflection training data. Then, we compare the effect of different types of noise on multiple state-of-the-art inflection models. Finally, we propose a novel character-level masked language modeling (CMLM) pretraining objective and explore its impact on the models' resistance to noise. Our experiments show that various architectures are impacted differently by separate types of noise, but encoder-decoders tend to be more robust to noise than models trained with a copy bias. CMLM pretraining helps transformers, but has lower impact on LSTMs.