# High Quality ELMo Embeddings for Seven Less-Resourced Languages

arXiv: https://arxiv.org/abs/1911.10049 · HTML: https://mlsanity.com/paper/1911.10049/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2019.

## Built on by (most cited first, 5 of 5)

- [Large-Scale Contextualised Language Modelling for Norwegian](https://mlsanity.com/paper/2104.06546/) (2021)
- [Developing and Evaluating Tiny to Medium-Sized Turkish BERT Models](https://mlsanity.com/paper/2307.14134/) (2023)
- [The Current State of Finnish NLP](https://mlsanity.com/paper/2109.11326/) (2021)
- [Evaluation of contextual embeddings on less-resourced languages](https://mlsanity.com/paper/2107.10614/) (2021)
- [Cross-lingual alignments of ELMo contextual embeddings](https://mlsanity.com/paper/2106.15986/) (2021)

## Abstract (arXiv)

Recent results show that deep neural networks using contextual embeddings significantly outperform non-contextual embeddings on a majority of text classification task. We offer precomputed embeddings from popular contextual ELMo model for seven languages: Croatian, Estonian, Finnish, Latvian, Lithuanian, Slovenian, and Swedish. We demonstrate that the quality of embeddings strongly depends on the size of training set and show that existing publicly available ELMo embeddings for listed languages shall be improved. We train new ELMo embeddings on much larger training sets and show their advantage over baseline non-contextual FastText embeddings. In evaluation, we use two benchmarks, the analogy task and the NER task.