mlsanityData to 14 Oct 2026

PL-MTEB: Polish Massive Text Embedding Benchmark

arXiv 2405.10138 ↗First posted 16 May 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce the Polish Massive Text Embedding Benchmark (PL-MTEB), a comprehensive benchmark for text embeddings in the Polish language. PL-MTEB comprises 30 diverse NLP tasks across five categories: classification, clustering, pair classification, information retrieval, and semantic text similarity. Within the scope of this work, we added 12 new Polish-language tasks to MTEB based on existing datasets and prepared two new datasets used to create four clustering tasks. We evaluated 30 publicly available text embedding models, including Polish and multilingual models. We analyzed the results in detail for specific task types and model sizes. We made the prepared datasets, the source code for evaluation, and the obtained results available to the public at https://github.com/rafalposwiata/pl-mteb.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 33 of its 72 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

33 of 72 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  11. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  12. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
21 more
  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  2. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  3. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  4. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  5. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  6. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  7. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  8. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  9. 2020Embedding-based Retrieval in Facebook Searchcited by 156
  10. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  11. 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
  12. 2020KLEJ: Comprehensive Benchmark for Polish Language Understandingcited by 51
  13. 2021HerBERT: Efficiently Pretrained Transformer-based Language Model for Polishcited by 51
  14. 2020Pre-training Polish Transformer-based Language Models at Scalecited by 21
  15. 2019Evaluation of Sentence Representations in Polishcited by 18
  16. 2023BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Languagecited by 15
  17. 2024PIRB: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methodscited by 14
  18. 2022This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polishcited by 13
  19. 2022Training Effective Neural Sentence Encoders from Automatically Mined Paraphrasescited by 11
  20. 2023MAUPQA: Massive Automatically-created Polish Question Answering Datasetcited by 11
  21. 2023Silver Retriever: Advancing Neural Passage Retrieval for Polish Question Answeringcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 9; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  2. 2024Making Text Embedders Few-Shot Learnerscited by 79
  3. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  4. 2024Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarkscited by 10
  5. 2025FaMTEB: Massive Text Embedding Benchmark in Persian Languagecited by 10
  6. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  7. 2025HUME: Measuring the Human-Model Performance Gap in Text Embedding Taskscited by 7
  8. 2024BEIR-NL: Zero-shot Information Retrieval Benchmark for the Dutch Languagecited by 6
  9. 2025Benchmarking Information Retrieval Models on Complex Retrieval Taskscited by 6
  10. 2025MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutchcited by 5
  11. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  12. 2025AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languagescited by 3
7 more
  1. 2025PL-Guard: Benchmarking Language Model Safety for Polishcited by 2
  2. 2024A Comparative Study of Text Retrieval Models on DaReCzechcited by 0
  3. 2025Unveiling Dual Quality in Product Reviews: An NLP-Based Approachcited by 0
  4. 2026ML-Embed: Inclusive and Efficient Embeddings for a Multilingual Worldcited by 0
  5. 2026SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptationcited by 0
  6. 2026MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguesecited by 0
  7. 2026Dataset Scarcity Limits Robust Evaluation of Multilingual Embedding Models: A Case Study of Slavic Languagescited by 0