# Improving Lemmatization of Non-Standard Languages with Joint Learning

arXiv: https://arxiv.org/abs/1903.06939 · HTML: https://mlsanity.com/paper/1903.06939/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2019.

## Built on by (most cited first, 12 of 12)

- [Neural Morphology Dataset and Models for Multiple Languages, from the Large to the Endangered](https://mlsanity.com/paper/2105.12428/) (2021)
- [From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French](https://mlsanity.com/paper/2202.09452/) (2022)
- [Corpus and Models for Lemmatisation and POS-tagging of Old French](https://mlsanity.com/paper/2109.11442/) (2021)
- [Computational Morphology with Neural Network Approaches](https://mlsanity.com/paper/2105.09404/) (2021)
- [Handling Heavily Abbreviated Manuscripts: HTR engines vs text normalisation approaches](https://mlsanity.com/paper/2107.03450/) (2021)
- [Lexicon and Rule-based Word Lemmatization Approach for the Somali Language](https://mlsanity.com/paper/2308.01785/) (2023)
- [Modern Models, Medieval Texts: A POS Tagging Study of Old Occitan](https://mlsanity.com/paper/2503.07827/) (2025)
- [Detecting Sexual Content at the Sentence Level in First Millennium Latin Texts](https://mlsanity.com/paper/2309.14974/) (2023)
- [Under-resourced studies of under-resourced languages: lemmatization and POS-tagging with LLM annotators for historical Armenian, Georgian, Greek and Syriac](https://mlsanity.com/paper/2602.15753/) (2026)
- [The Patrologia Graeca Corpus: OCR, Annotation, and Open Release of Noisy Nineteenth-Century Polytonic Greek Editions](https://mlsanity.com/paper/2603.09470/) (2026)
- [From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages](https://mlsanity.com/paper/2605.09147/) (2026)
- [Lost in Translation? Exploring the Shift in Grammatical Gender from Latin to Occitan](https://mlsanity.com/paper/2605.09156/) (2026)

## Abstract (arXiv)

Lemmatization of standard languages is concerned with (i) abstracting over morphological differences and (ii) resolving token-lemma ambiguities of inflected words in order to map them to a dictionary headword. In the present paper we aim to improve lemmatization performance on a set of non-standard historical languages in which the difficulty is increased by an additional aspect (iii): spelling variation due to lacking orthographic standards. We approach lemmatization as a string-transduction task with an encoder-decoder architecture which we enrich with sentence context information using a hierarchical sentence encoder. We show significant improvements over the state-of-the-art when training the sentence encoder jointly for lemmatization and language modeling. Crucially, our architecture does not require POS or morphological annotations, which are not always available for historical corpora. Additionally, we also test the proposed model on a set of typologically diverse standard languages showing results on par or better than a model without enhanced sentence representations and previous state-of-the-art systems. Finally, to encourage future work on processing of non-standard varieties, we release the dataset of non-standard languages underlying the present study, based on openly accessible sources.