# UDPipe at SIGMORPHON 2019: Contextualized Embeddings, Regularization with Morphological Categories, Corpora Merging

arXiv: https://arxiv.org/abs/1908.06931 · HTML: https://mlsanity.com/paper/1908.06931/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2019.

## Built on by (most cited first, 15 of 15)

- [Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation](https://mlsanity.com/paper/2209.15236/) (2022)
- [Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy](https://mlsanity.com/paper/2106.13553/) (2021)
- [Morphological Processing of Low-Resource Languages: Where We Are and What's Next](https://mlsanity.com/paper/2203.08909/) (2022)
- [Tokenization with Factorized Subword Encoding](https://mlsanity.com/paper/2306.07764/) (2023)
- [On the Role of Morphological Information for Contextual Lemmatization](https://mlsanity.com/paper/2302.00407/) (2023)
- [Computational Morphology with Neural Network Approaches](https://mlsanity.com/paper/2105.09404/) (2021)
- [Recent advancements in computational morphology : A comprehensive survey](https://mlsanity.com/paper/2406.05424/) (2024)
- [Evaluating Shortest Edit Script Methods for Contextual Lemmatization](https://mlsanity.com/paper/2403.16968/) (2024)
- [Comparison of Current Approaches to Lemmatization: A Case Study in Estonian](https://mlsanity.com/paper/2404.15003/) (2024)
- [Enhancing Sequence-to-Sequence Neural Lemmatization with External Resources](https://mlsanity.com/paper/2101.12056/) (2021)
- [Understanding the effects of word-level linguistic annotations in under-resourced neural machine translation](https://mlsanity.com/paper/2401.16078/) (2024)
- [Lemma Dilemma: On Lemma Generation Without Domain- or Language-Specific Training Data](https://mlsanity.com/paper/2510.07434/) (2025)
- [Prague Dependency Treebank -- Consolidated 2.0: Enriching a Complex Annotation Scheme](https://mlsanity.com/paper/2606.24324/) (2026)
- [Meet UD_Czech-PDTC: A Large and Genre-Rich Treebank in Universal Dependencies](https://mlsanity.com/paper/2606.24337/) (2026)
- [Ossetic-COT: Designing a morphologically annotated corpus and morphological analyzer for Ossetic](https://mlsanity.com/paper/2607.04895/) (2026)

## Abstract (arXiv)

We present our contribution to the SIGMORPHON 2019 Shared Task: Crosslinguality and Context in Morphology, Task 2: contextual morphological analysis and lemmatization. We submitted a modification of the UDPipe 2.0, one of best-performing systems of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies and an overall winner of the The 2018 Shared Task on Extrinsic Parser Evaluation. As our first improvement, we use the pretrained contextualized embeddings (BERT) as additional inputs to the network; secondly, we use individual morphological features as regularization; and finally, we merge the selected corpora of the same language. In the lemmatization task, our system exceeds all the submitted systems by a wide margin with lemmatization accuracy 95.78 (second best was 95.00, third 94.46). In the morphological analysis, our system placed tightly second: our morphological analysis accuracy was 93.19, the winning system's 93.23.