# The Paradigm Discovery Problem

arXiv: https://arxiv.org/abs/2005.01630 · HTML: https://mlsanity.com/paper/2005.01630/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2020.

## Built on by (most cited first, 6 of 6)

- [Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model](https://mlsanity.com/paper/2310.15113/) (2023)
- [Morphological Processing of Low-Resource Languages: Where We Are and What's Next](https://mlsanity.com/paper/2203.08909/) (2022)
- [CaMEL: Case Marker Extraction without Labels](https://mlsanity.com/paper/2203.10010/) (2022)
- [An Investigation of Noise in Morphological Inflection](https://mlsanity.com/paper/2305.16581/) (2023)
- [Minimal Supervision for Morphological Inflection](https://mlsanity.com/paper/2104.08512/) (2021)
- [Can a Neural Model Guide Fieldwork? A Case Study on Morphological Data Collection](https://mlsanity.com/paper/2409.14628/) (2024)

## Abstract (arXiv)

This work treats the paradigm discovery problem (PDP), the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work. Our code and data are available for public use.