# Evaluating Contextualized Embeddings on 54 Languages in POS Tagging, Lemmatization and Dependency Parsing

arXiv: https://arxiv.org/abs/1908.07448 · HTML: https://mlsanity.com/paper/1908.07448/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2019.

## Built on by (most cited first, 8 of 8)

- [Exploring Large Language Models for Classical Philology](https://mlsanity.com/paper/2305.13698/) (2023)
- [Understanding Model Robustness to User-generated Noisy Texts](https://mlsanity.com/paper/2110.07428/) (2021)
- [Lessons learned from the evaluation of Spanish Language Models](https://mlsanity.com/paper/2212.08390/) (2022)
- [Head-driven Phrase Structure Parsing in O(n3) Time Complexity](https://mlsanity.com/paper/2105.09835/) (2021)
- [Revisiting Tri-training of Dependency Parsers](https://mlsanity.com/paper/2109.08122/) (2021)
- [Heidelberg-Boston @ SIGTYP 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers](https://mlsanity.com/paper/2405.20145/) (2024)
- [Latin Treebanks in Review: An Evaluation of Morphological Tagging Across Time](https://mlsanity.com/paper/2408.06675/) (2024)
- [Thai Universal Dependency Treebank](https://mlsanity.com/paper/2405.07586/) (2024)

## Abstract (arXiv)

We present an extensive evaluation of three recently proposed methods for contextualized embeddings on 89 corpora in 54 languages of the Universal Dependencies 2.3 in three tasks: POS tagging, lemmatization, and dependency parsing. Employing the BERT, Flair and ELMo as pretrained embedding inputs in a strong baseline of UDPipe 2.0, one of the best-performing systems of the CoNLL 2018 Shared Task and an overall winner of the EPE 2018, we present a one-to-one comparison of the three contextualized word embedding methods, as well as a comparison with word2vec-like pretrained embeddings and with end-to-end character-level word embeddings. We report state-of-the-art results in all three tasks as compared to results on UD 2.2 in the CoNLL 2018 Shared Task.