mlsanityData to 14 Oct 2026

MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch

arXiv 2509.12340 ↗First posted 15 Sep 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Recently, embedding resources, including models, benchmarks, and datasets, have been widely released to support a variety of languages. However, the Dutch language remains underrepresented, typically comprising only a small fraction of the published multilingual resources. To address this gap and encourage the further development of Dutch embeddings, we introduce new resources for their evaluation and generation. First, we introduce the Massive Text Embedding Benchmark for Dutch (MTEB-NL), which includes both existing Dutch datasets and newly created ones, covering a wide range of tasks. Second, we provide a training dataset compiled from available Dutch retrieval datasets, complemented with synthetic data generated by large language models to expand task coverage beyond retrieval. Finally, we release a series of E5-NL models compact yet efficient embedding models that demonstrate strong performance across multiple tasks. We make our resources publicly available through the Hugging Face Hub and the MTEB package.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 63 of its 100 references that are in the map. First posted on arXiv 15 Sep 2025.

Builds on

63 of 100 references.

References · 63

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  9. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  10. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  12. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
38 more
  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  2. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  3. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  5. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  6. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  7. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  8. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  9. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  10. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  11. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  12. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  13. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  14. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  15. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  16. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 141
  17. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  18. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  19. 2019BERTje: A Dutch BERT Modelcited by 109
  20. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 90
  21. 2020RobBERT: a Dutch RoBERTa-based Language Modelcited by 88
  22. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
  23. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  24. 2020Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Societycited by 72
  25. 2024Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Modelscited by 61
  26. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  27. 2020XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualizationcited by 42
  28. 2021A Statutory Article Retrieval Dataset in Frenchcited by 40
  29. 2024Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPcited by 36
  30. 2025KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Modelcited by 36
  31. 2024Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Taskscited by 34
  32. 2024The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embeddingcited by 26
  33. 2021SICKNL: A Dataset for Dutch Natural Language Inferencecited by 20
  34. 2023An Efficient Multilingual Language Model Compression through Vocabulary Trimmingcited by 20
  35. 2024The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model designcited by 20
  36. 2024PL-MTEB: Polish Massive Text Embedding Benchmarkcited by 19
  37. 2023BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Languagecited by 15
  38. 2019The merits of Universal Language Model Fine-tuning for Small Datasets -- a case with Dutch book reviewscited by 14

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026ML-Embed: Inclusive and Efficient Embeddings for a Multilingual Worldcited by 0
  2. 2026Modeling semantic association in self-paced reading with language model embeddingscited by 0
  3. 2026HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditionscited by 0
  4. 2026MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguesecited by 0
  5. 2026Do Static Embeddings Add Value to Hybrid Dutch Retrieval?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
  2. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmark
  3. 2025EmbeddingGemma: Powerful and Lightweight Text Representations
  4. 2026F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World
  5. 2026Less is More: Adapting Text Embeddings for Low-Resource Languages with Small Scale Noisy Synthetic Data
  6. 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
  7. 2025AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages
  8. 2026Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems