# Falling Through the Gaps: Neural Architectures as Models of Morphological Rule Learning

arXiv: https://arxiv.org/abs/2105.03710 · HTML: https://mlsanity.com/paper/2105.03710/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 33 references that are in the map.
- First posted on arXiv 8 May 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate](https://mlsanity.com/paper/1807.04783/) (2018)
- [Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection](https://mlsanity.com/paper/1906.01280/) (2019)
- [Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German plurals](https://mlsanity.com/paper/2005.08826/) (2020)

## Built on by (most cited first, 1 of 1)

- [How do we get there? Evaluating transformer neural networks as cognitive models for English past tense inflection](https://mlsanity.com/paper/2210.09167/) (2022)

## Abstract (arXiv)

Recent advances in neural architectures have revived the problem of morphological rule learning. We evaluate the Transformer as a model of morphological rule learning and compare it with Recurrent Neural Networks (RNN) on English, German, and Russian. We bring to the fore a hitherto overlooked problem, the morphological gaps, where the expected inflection of a word is missing. For example, 63 Russian verbs lack a first-person-singular present form such that one cannot comfortably say "*ovsvcuvsvcu" ("I feel"). Even English has gaps, such as the past participle of "stride": the function of morphological inflection can be partial. Both neural architectures produce inflections that ought to be missing. Analyses reveal that Transformers recapitulate the statistical distribution of inflections in the training data, similar to RNNs. Models' success on English and German is driven by the fact that rules in these languages can be identified with the majority forms, which is not universal.