mlsanityData to 14 Oct 2026

Rethinking Hybrid Retrieval: When Small Embeddings and LLM Re-ranking Beat Bigger Models

arXiv 2506.00049 ↗First posted 28 May 2025cs.IR cs.AIcited by 1

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

This paper presents a comparison of embedding models in tri-modal hybrid retrieval for Retrieval-Augmented Generation (RAG) systems. We investigate the fusion of dense semantic, sparse lexical, and graph-based embeddings, focusing on the performance of the MiniLM-v6 and BGE-Large architectures. Contrary to conventional assumptions, our results show that the compact MiniLM-v6 outperforms the larger BGE-Large when integrated with LLM-based re-ranking within our tri-modal hybrid framework. Experiments conducted on the SciFact, FIQA, and NFCorpus datasets demonstrate significant improvements in retrieval quality with the MiniLM-v6 configuration. The performance difference is particularly pronounced in agentic re-ranking scenarios, indicating better alignment between MiniLM-v6's embedding space and LLM reasoning. Our findings suggest that embedding model selection for RAG systems should prioritize compatibility with multi-signal fusion and LLM alignment, rather than relying solely on larger models. This approach may reduce computational requirements while improving retrieval accuracy and efficiency.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 24 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

20 of 24 references.

References · 20

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  6. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  8. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  9. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  10. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  11. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  12. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approachcited by 119
8 more
  1. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  2. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  3. 2024Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrieverscited by 84
  4. 2024Generative Multi-Modal Knowledge Retrieval with Large Language Modelscited by 22
  5. 2024Combining Knowledge Graphs and Large Language Modelscited by 16
  6. 2022Combining Static and Contextualised Multilingual Embeddingscited by 8
  7. 2025DAT: Dynamic Alpha Tuning for Hybrid Retrieval in Retrieval-Augmented Generationcited by 7
  8. 2024ChatGPT Alternative Solutions: Large Language Models Surveycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026RAGnaroX: A Secure, Local-Hosted ChatOps Assistant Using Small Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ELITE: Embedding-Less retrieval with Iterative Text Exploration
  2. 2024Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systems
  3. 2025Re-ranking the Context for Multimodal Retrieval Augmented Generation
  4. 2025Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts
  5. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  6. 2024REFINE on Scarce Data: Retrieval Enhancement through Fine-Tuning via Model Fusion of Embedding Models
  7. 2026EmbeddingRWKV: State-Centric Retrieval with Reusable States
  8. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark