# When Irregularity Helps: A Subclass Analysis of Inductive Bias in Neural Morphology

arXiv: https://arxiv.org/abs/2605.20558 · HTML: https://mlsanity.com/paper/2605.20558/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 19 May 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [UniMorph 2.0: Universal Morphology](https://mlsanity.com/paper/1810.11101/) (2018)
- [The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection](https://mlsanity.com/paper/1810.07125/) (2018)
- [CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages](https://mlsanity.com/paper/1706.09031/) (2017)
- [SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection](https://mlsanity.com/paper/2006.11572/) (2020)
- [Exact Hard Monotonic Attention for Character-Level Transduction](https://mlsanity.com/paper/1905.06319/) (2019)
- [Morphological Inflection Generation Using Character Sequence to Sequence Learning](https://mlsanity.com/paper/1512.06110/) (2015)
- [Morphological Irregularity Correlates with Frequency](https://mlsanity.com/paper/1906.11483/) (2019)
- [Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection](https://mlsanity.com/paper/1606.00589/) (2016)
- [Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers](https://mlsanity.com/paper/2410.21013/) (2024)
- [Searching for Search Errors in Neural Morphological Inflection](https://mlsanity.com/paper/2102.08424/) (2021)
- [Character-aware Transformers Learn an Irregular Morphological Pattern Yet None Generalize Like Humans](https://mlsanity.com/paper/2602.14100/) (2026)

## Built on by (most cited first, 1 of 1)

- [Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection](https://mlsanity.com/paper/2609.21179/) (2026)

## Abstract (arXiv)

Neural morphological generation systems often achieve high aggregate accuracy on benchmark datasets, yet such performance can conceal systematic errors concentrated in rare morphological subclasses. We examine Japanese past-tense verb inflection and show that a very small, structurally specific irregular subtype (<1% of data) accounts for a disproportionate share of model errors. Controlled ablation experiments demonstrate that removing this subtype yields larger improvements in generalization than removing all irregular verbs, indicating that not all irregularity contributes equally to model instability. These findings suggest that error concentration is driven by the interaction between extreme low-frequency morphological patterns and specific morphophonological processes, particularly gemination. We argue that morphological evaluation should incorporate finer-grained subclass analysis beyond standard conjugation categories.