mlsanityData to 14 Oct 2026

F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World

arXiv 2603.19223 ↗First posted 19 Mar 2026cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

We present F2LLM-v2, a new family of general-purpose, multilingual embedding models in 8 distinct sizes ranging from 80M to 14B. Trained on a newly curated composite of 60 million publicly available high-quality data samples, F2LLM-v2 supports more than 200 languages, with a particular emphasis on previously underserved mid- and low-resource languages. By integrating a two-stage LLM-based embedding training pipeline with matryoshka learning, model pruning, and knowledge distillation techniques, we present models that are far more efficient than previous LLM-based embedding models while retaining competitive performances. Extensive evaluations confirm that F2LLM-v2-14B ranks first on 11 MTEB benchmarks, while the smaller models in the family also set a new state of the art for resource-constrained applications. To facilitate open-source embedding model research, we release all models, data, code, and intermediate checkpoints.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 87 of its 132 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

87 of 132 references.

References · 87

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  12. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
38 more
  1. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  2. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  3. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  6. 2015Teaching Machines to Read and Comprehendcited by 1,049
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  8. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  9. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  11. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  12. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  13. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  14. 2019ELI5: Long Form Question Answeringcited by 575
  15. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  16. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  17. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  18. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  19. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  20. 2023Improving Text Embeddings with Large Language Modelscited by 303
  21. 2022Matryoshka Representation Learningcited by 282
  22. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  23. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  24. 2024Generative Representational Instruction Tuningcited by 233
  25. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  26. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  27. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
  28. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 153
  29. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  30. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  31. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 139
  32. 2019A Question-Entailment Approach to Question Answeringcited by 134
  33. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  34. 2020MLSUM: The Multilingual Summarization Corpuscited by 121
  35. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 119
  36. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  37. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  38. 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 90

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model
  2. 2025xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation
  3. 2026Bootstrapping Embeddings for Low Resource Languages
  4. 2025Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders
  5. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data
  6. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias
  7. 2024Multi-Lingual Malaysian Embedding: Leveraging Large Language Models for Semantic Representations
  8. 2025Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks