ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World
Not on the map. Why?
Abstract (arXiv)
The development of high-quality text embeddings is increasingly drifting toward an exclusionary future, defined by three critical barriers: prohibitive computational costs, a narrow linguistic focus that neglects most of the world's languages, and a lack of transparency from closed-source or open-weight models that stifles research. To dismantle these barriers, we introduce ML-Embed, a suite of inclusive and efficient models built upon a new framework: 3-Dimensional Matryoshka Learning (3D-ML). Our framework addresses the computational challenge with comprehensive efficiency across the entire model lifecycle. Beyond the storage benefits of Matryoshka Representation Learning (MRL) and flexible inference-time depth provided by Matryoshka Layer Learning (MLL), we introduce Matryoshka Embedding Learning (MEL) for enhanced parameter efficiency. To address the linguistic challenge, we curate a massively multilingual dataset and train a suite of models ranging from 140M to 8B parameters. In a direct commitment to transparency, we release all models, data, and code. Extensive evaluation on 430 tasks demonstrates that our models set new records on 9 of 17 evaluated MTEB benchmarks, with particularly strong results in low-resource languages, providing a reproducible blueprint for building globally equitable and computationally efficient AI systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 86 of its 131 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
86 of 131 references.
References · 86
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2025Qwen3 Technical Reportcited by 8,820
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
38 more
- 2015A large annotated corpus for learning natural language inferencecited by 1,847
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2015Teaching Machines to Read and Comprehendcited by 1,049
- 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
- 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
- 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
- 2022MTEB: Massive Text Embedding Benchmarkcited by 640
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
- 2019ELI5: Long Form Question Answeringcited by 575
- 2020Fact or Fiction: Verifying Scientific Claimscited by 554
- 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
- 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
- 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
- 2023Improving Text Embeddings with Large Language Modelscited by 303
- 2022Matryoshka Representation Learningcited by 282
- 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
- 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
- 2024Generative Representational Instruction Tuningcited by 233
- 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
- 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
- 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
- 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 153
- 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
- 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
- 2020MLSUM: The Multilingual Summarization Corpuscited by 121
- 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 119
- 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
- 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
- 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 90
- 2020OCNLI: Original Chinese Natural Language Inferencecited by 86
- 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 81
50 most cited of 86 All 86 (JSON)
Built on by
Not cited yet.