mlsanityData to 7 Oct 2026

Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model

arXiv 2507.05513 ↗First posted 7 Jul 2025cs.CV cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

Motivated by the growing demand for retrieval systems that operate across modalities, we introduce llama-nemoretriever-colembed, a unified text-image retrieval model that delivers state-of-the-art performance across multiple benchmarks. We release two model variants, 1B and 3B. The 3B model achieves state of the art performance, scoring NDCG@5 91.0 on ViDoRe V1 and 63.5 on ViDoRe V2, placing first on both leaderboards as of June 27, 2025. Our approach leverages the NVIDIA Eagle2 Vision-Language model (VLM), modifies its architecture by replacing causal attention with bidirectional attention, and integrates a ColBERT-style late interaction mechanism to enable fine-grained multimodal retrieval in a shared embedding space. While this mechanism delivers superior retrieval accuracy, it introduces trade-offs in storage and efficiency. We provide a comprehensive analysis of these trade-offs. Additionally, we adopt a two-stage training strategy to enhance the model's retrieval capabilities.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

21 of 27 references.

References · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  7. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  8. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  9. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  10. 2023Improving Text Embeddings with Large Language Modelscited by 295
  11. 2023AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into Onecited by 165
  12. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
9 more
  1. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
  2. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  3. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  4. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 58
  5. 2024MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodingscited by 30
  6. 2025Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modalitycited by 30
  7. 2024Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAGcited by 16
  8. 2025MIRACL-VISION: A Large, multilingual, visual document retrieval benchmarkcited by 9
  9. 2025Maintaining MTEB: Towards Long Term Usability and Reproducibility of Embedding Benchmarkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 16.

24 papers.

Sort:

By area:

  1. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  2. 2025Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Taskscited by 46
  3. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 36
  4. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 22
  5. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
  6. 2025ModernVBERT: Towards Smaller Visual Document Retrieverscited by 19
  7. 2025Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model for Text, Image, Audio, and Videocited by 10
  8. 2025Guided Query Refinement: Multimodal Hybrid Retrieval with Test-Time Optimizationcited by 10
  9. 2026Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrievalcited by 8
  10. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
  11. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  12. 2026Multi-Vector Index Compression in Any Modalitycited by 8
12 more
  1. 2026Structural Anchor Pruning: Training-Free Multi-Vector Compression for Visual Document Retrievalcited by 6
  2. 2026CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embeddingcited by 6
  3. 2026Learning Retrieval Models with Sparse Autoencoderscited by 5
  4. 2025Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracycited by 4
  5. 2025Attention Grounded Enhancement for Visual Document Retrievalcited by 3
  6. 2026Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representationscited by 3
  7. 2026MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Frameworkcited by 2
  8. 2026LEMUR: Learned Multi-Vector Retrievalcited by 1
  9. 2026Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrievalcited by 0
  10. 2026Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrievalcited by 0
  11. 2026ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAGcited by 0
  12. 2026KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024What If We Recaption Billions of Web Images with LLaMA-3?
  2. 2024INQUIRE: A Natural World Text-to-Image Retrieval Benchmark
  3. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant
  4. 2024Benchmark Granularity and Model Robustness for Image-Text Retrieval
  5. 2026MASCOT: Model-Aware Submodular Coverage for Composite-Attribute Text-to-Image Retrieval
  6. 2026Indexing Multimodal Language Models for Large-scale Image Retrieval
  7. 2024TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Models
  8. 2026Unified Multimodal and Multilingual Retrieval via Multi-Task Learning with NLU Integration