# Probing Subphonemes in Morphology Models

arXiv: https://arxiv.org/abs/2505.11297 · HTML: https://mlsanity.com/paper/2505.11297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Using the Output Embedding to Improve Language Models](https://mlsanity.com/paper/1608.05859/) (2016)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection](https://mlsanity.com/paper/1910.11493/) (2019)
- [Applying the Transformer to Character-level Transduction](https://mlsanity.com/paper/2005.10213/) (2020)
- [CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages](https://mlsanity.com/paper/1706.09031/) (2017)
- [Pushing the Limits of Low-Resource Morphological Inflection](https://mlsanity.com/paper/1908.05838/) (2019)
- [(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance](https://mlsanity.com/paper/2108.05682/) (2021)
- [Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models](https://mlsanity.com/paper/2104.06483/) (2021)
- [Information-Theoretic Characterization of Vowel Harmony: A Cross-Linguistic Study on Word Lists](https://mlsanity.com/paper/2308.04885/) (2023)
- [An Investigation of Noise in Morphological Inflection](https://mlsanity.com/paper/2305.16581/) (2023)
- [Exploring Linguistic Probes for Morphological Generalization](https://mlsanity.com/paper/2310.13686/) (2023)

## Abstract (arXiv)

Transformers have achieved state-of-the-art performance in morphological inflection tasks, yet their ability to generalize across languages and morphological rules remains limited. One possible explanation for this behavior can be the degree to which these models are able to capture implicit phenomena at the phonological and subphonemic levels. We introduce a language-agnostic probing method to investigate phonological feature encoding in transformers trained directly on phonemes, and perform it across seven morphologically diverse languages. We show that phonological features which are local, such as final-obstruent devoicing in Turkish, are captured well in phoneme embeddings, whereas long-distance dependencies like vowel harmony are better represented in the transformer's encoder. Finally, we discuss how these findings inform empirical strategies for training morphological models, particularly regarding the role of subphonemic feature acquisition.