mlsanityData to 14 Oct 2026

F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Data

arXiv 2510.02294 ↗First posted 2 Oct 2025cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce F2LLM - Foundation to Feature Large Language Models, a suite of state-of-the-art embedding models in three sizes: 0.6B, 1.7B, and 4B. Unlike previous top-ranking embedding models that require massive contrastive pretraining, sophisticated training pipelines, and costly synthetic training data, F2LLM is directly finetuned from foundation models on 6 million query-document-negative tuples curated from open-source, non-synthetic datasets, striking a strong balance between training cost, model size, and embedding performance. On the MTEB English leaderboard, F2LLM-4B ranks 2nd among models with approximately 4B parameters and 7th overall, while F2LLM-1.7B ranks 1st among models in the 1B-2B size range. To facilitate future research in the field, we release the models, training dataset, and code, positioning F2LLM as a strong, reproducible, and budget-friendly baseline for future works.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 44 of its 68 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

44 of 68 references.

References · 44

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  3. 2023Mistral 7Bcited by 3,164
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
32 more
  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  2. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  3. 2015Teaching Machines to Read and Comprehendcited by 1,049
  4. 2018Neural Network Acceptability Judgmentscited by 958
  5. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  6. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  8. 2019ELI5: Long Form Question Answeringcited by 575
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  10. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  11. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  12. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  13. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  14. 2023Improving Text Embeddings with Large Language Modelscited by 303
  15. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  16. 2024Generative Representational Instruction Tuningcited by 233
  17. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  18. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
  19. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  20. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  21. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 119
  22. 2023Scaling Sentence Embeddings with Large Language Modelscited by 85
  23. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 83
  24. 2024Making Text Embedders Few-Shot Learnerscited by 79
  25. 2024Repetition Improves Language Model Embeddingscited by 72
  26. 2019Natural Language Understanding with the Quora Question Pairs Datasetcited by 51
  27. 2021I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviewscited by 45
  28. 2019AmazonQA: A Review-Based Question Answering Taskcited by 43
  29. 2021TWEAC: Transformer with Extendable QA Agent Classifierscited by 37
  30. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 36
  31. 2024D2LLM: Decomposed and Distilled Large Language Models for Semantic Searchcited by 9
  32. 2025LGAI-EMBEDDING-Preview Technical Reportcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual Worldcited by 12
  2. 2026StoryScope: Investigating idiosyncrasies in AI fictioncited by 9
  3. 2026BROTHER: Behavioral Recognition Optimized Through Heterogeneous Ensemble Regularization for Ambivalence and Hesitancycited by 7
  4. 2025C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Poolingcited by 3
  5. 2026Beyond Retrieval: A Multitask Benchmark and Model for Code Searchcited by 1
  6. 2026ML-Embed: Inclusive and Efficient Embeddings for a Multilingual Worldcited by 0
  7. 2026Multimodal Ambivalence and Hesitancy Recognition via Cross-Attention and Gated Fusioncited by 0
  8. 2026Reasoning Quality Matters: Combating Reasoning Collapse in LLM-based Embedding Learningcited by 0