# Evaluating Shortest Edit Script Methods for Contextual Lemmatization

arXiv: https://arxiv.org/abs/2403.16968 · HTML: https://mlsanity.com/paper/2403.16968/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 52 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language](https://mlsanity.com/paper/1905.07213/) (2019)
- [Spanish Pre-trained BERT Model and Evaluation Data](https://mlsanity.com/paper/2308.02976/) (2023)
- [HerBERT: Efficiently Pretrained Transformer-based Language Model for Polish](https://mlsanity.com/paper/2105.01735/) (2021)
- [The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection](https://mlsanity.com/paper/1910.11493/) (2019)
- [Give your Text Representation Models some Love: the Case for Basque](https://mlsanity.com/paper/2004.00033/) (2020)
- [UDPipe at SIGMORPHON 2019: Contextualized Embeddings, Regularization with Morphological Categories, Corpora Merging](https://mlsanity.com/paper/1908.06931/) (2019)
- [Robust Multilingual Named Entity Recognition with Shallow Semi-Supervised Features](https://mlsanity.com/paper/1701.09123/) (2017)
- [Lessons learned from the evaluation of Spanish Language Models](https://mlsanity.com/paper/2212.08390/) (2022)
- [A Simple Joint Model for Improved Contextual Neural Lemmatization](https://mlsanity.com/paper/1904.02306/) (2019)

## Built on by (most cited first, 1 of 1)

- [GliLem: Leveraging GliNER for Contextualized Lemmatization in Estonian](https://mlsanity.com/paper/2412.20597/) (2024)

## Abstract (arXiv)

Modern contextual lemmatizers often rely on automatically induced Shortest Edit Scripts (SES), namely, the number of edit operations to transform a word form into its lemma. In fact, different methods of computing SES have been proposed as an integral component in the architecture of several state-of-the-art contextual lemmatizers currently available. However, previous work has not investigated the direct impact of SES in the final lemmatization performance. In this paper we address this issue by focusing on lemmatization as a token classification task where the only input that the model receives is the word-label pairs in context, where the labels correspond to previously induced SES. Thus, by modifying in our lemmatization system only the SES labels that the model needs to learn, we may then objectively conclude which SES representation produces the best lemmatization results. We experiment with seven languages of different morphological complexity, namely, English, Spanish, Basque, Russian, Czech, Turkish and Polish, using multilingual and language-specific pre-trained masked language encoder-only models as a backbone to build our lemmatizers. Comprehensive experimental results, both in- and out-of-domain, indicate that computing the casing and edit operations separately is beneficial overall, but much more clearly for languages with high-inflected morphology. Notably, multilingual pre-trained language models consistently outperform their language-specific counterparts in every evaluation setting.