mlsanityData to 13 Oct 2026

MTEB: Massive Text Embedding Benchmark

arXiv 2210.07316 ↗First posted 13 Oct 2022cs.CL cs.IR cs.LGcited by 630

Not on the map. Why?

Cited by 79 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other tasks. It is unclear whether state-of-the-art embeddings on semantic textual similarity (STS) can be equally well applied to other tasks like clustering or reranking. This makes progress in the field difficult to track, as various models are constantly being proposed without proper evaluation. To solve this problem, we introduce the Massive Text Embedding Benchmark (MTEB). MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages. Through the benchmarking of 33 models on MTEB, we establish the most comprehensive benchmark of text embeddings to date. We find that no particular text embedding method dominates across all tasks. This suggests that the field has yet to converge on a universal text embedding method and scale it up sufficiently to provide state-of-the-art results on all embedding tasks. MTEB comes with open-source code and a public leaderboard at https://github.com/embeddings-benchmark/mteb.

Key facts. Cited by 630 papers in the map (data through 13 Oct 2026). Builds on 43 of its 80 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

43 of 80 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,339
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,880
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,170
  10. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,985
  11. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,725
  12. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,551
31 more
  1. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,435
  2. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,409
  3. 2021Improving language models by retrieving from trillions of tokenscited by 1,286
  4. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,237
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 976
  6. 2020SummEval: Re-evaluating Summarization Evaluationcited by 668
  7. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 651
  8. 2020Language-agnostic BERT Sentence Embeddingcited by 622
  9. 2021Datasets: A Community Library for Natural Language Processingcited by 529
  10. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 487
  11. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  12. 2021Large Dual Encoders Are Generalizable Retrieverscited by 455
  13. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 431
  14. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 429
  15. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 425
  16. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  17. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  18. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  19. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  20. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 157
  21. 2020Embedding-based Retrieval in Facebook Searchcited by 155
  22. 2020Top2Vec: Distributed Representations of Topicscited by 125
  23. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  24. 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
  25. 2017A Continuously Growing Dataset of Sentential Paraphrasescited by 72
  26. 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 50
  27. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  28. 2021I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviewscited by 44
  29. 2021TWEAC: Transformer with Extendable QA Agent Classifierscited by 37
  30. 2018Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddingscited by 35
  31. 2018Adversarial Domain Adaptation for Duplicate Question Detectioncited by 30

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 6; 2023: 37; 2024: 161; 2025: 218; 2026: 208.

630 papers.

Sort:

By area:

  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,978
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,229
  3. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,222
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 714
  5. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 683
  6. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 467
  7. 2023Scaling Data-Constrained Language Modelscited by 398
  8. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 398
  9. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 357
  10. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  11. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 330
  12. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 298
38 more
  1. 2023Improving Text Embeddings with Large Language Modelscited by 297
  2. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 273
  3. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 241
  4. 2024Generative Representational Instruction Tuningcited by 229
  5. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 224
  6. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 193
  7. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  8. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 190
  9. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 184
  10. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 179
  11. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 160
  12. 2024Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrencecited by 156
  13. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 156
  14. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  15. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 132
  16. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 126
  17. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 112
  18. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 100
  19. 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
  20. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
  21. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  22. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 80
  23. 2024Making Text Embedders Few-Shot Learnerscited by 79
  24. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  25. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
  26. 2024Repetition Improves Language Model Embeddingscited by 71
  27. 2025Rank1: Test-Time Compute for Reranking in Information Retrievalcited by 69
  28. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 68
  29. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 67
  30. 2023RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Modelscited by 64
  31. 2024Human-inspired Episodic Memory for Infinite Context LLMscited by 62
  32. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  33. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  34. 2024Arctic-Embed 2.0: Multilingual Retrieval Without Compromisecited by 58
  35. 2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
  36. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 50
  37. 2025Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coincited by 49
  38. 2025Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Taskscited by 48

50 most cited of 630 All 630 (JSON)