mlsanityData to 14 Oct 2026

Granite Embedding Models

arXiv 2502.20204 ↗First posted 27 Feb 2025cs.IR cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce the Granite Embedding models, a family of encoder-based embedding models designed for retrieval tasks, spanning dense-retrieval and sparse retrieval architectures, with both English and Multilingual capabilities. This report provides the technical details of training these highly effective 12 layer embedding models, along with their efficient 6 layer distilled counterparts. Extensive evaluations show that the models, developed with techniques like retrieval oriented pretraining, contrastive finetuning, knowledge distillation, and model merging significantly outperform publicly available models of similar sizes on both internal IBM retrieval and search tasks, and have equivalent performance on widely used information retrieval benchmarks, while being trained on high-quality data suitable for enterprise use. We publicly release all our Granite Embedding models under the Apache 2.0 license, allowing both research and commercial use at https://huggingface.co/collections/ibm-granite.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

39 of 50 references.

References · 39

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  8. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  10. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
27 more
  1. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  2. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  3. 2018Born Again Neural Networkscited by 546
  4. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  5. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  6. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  7. 2023Improving Text Embeddings with Large Language Modelscited by 303
  8. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  9. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 265
  10. 2020MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformerscited by 245
  11. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  12. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 225
  13. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  14. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  15. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 127
  16. 2020Top2Vec: Distributed Representations of Topicscited by 126
  17. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  18. 2020Self-Knowledge Distillation with Progressive Refinement of Targetscited by 107
  19. 2022Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answeringcited by 72
  20. 2019Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrievalcited by 65
  21. 2023T2Ranking: A large-scale Chinese Benchmark for Passage Rankingcited by 60
  22. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 60
  23. 2020SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrievalcited by 50
  24. 2020Minimizing FLOPs to Learn Efficient Sparse Representationscited by 46
  25. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45
  26. 2023LM-Cocktail: Resilient Tuning of Language Models via Model Mergingcited by 44
  27. 2022DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Enginecited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2026CSRv2: Unlocking Ultra-Sparse Embeddingscited by 6
  2. 2025Search-R3: Unifying Reasoning and Embedding in Large Language Modelscited by 4
  3. 2025S-DAT: A Multilingual, GenAI-Driven Framework for Automated Divergent Thinking Assessmentcited by 3
  4. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  5. 2025DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Managementcited by 2
  6. 2026ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretrainingcited by 1
  7. 2025Beyond Traditional Diagnostics: Transforming Patient-Side Information into Predictive Insights with Knowledge Graphs and Prototypescited by 0
  8. 2026Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval
  2. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
  3. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms
  4. 2024Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systems
  5. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmark
  6. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval
  7. 2026Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems
  8. 2026UEmbed: Unified Sparse and Dense Multimodal Embeddings