# Sparse Meets Dense: A Hybrid Approach to Enhance Scientific Document Retrieval

arXiv: https://arxiv.org/abs/2401.04055 · HTML: https://mlsanity.com/paper/2401.04055/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2024.

## Built on by (most cited first, 16 of 16)

- [Human-inspired Perspectives: A Survey on AI Long-term Memory](https://mlsanity.com/paper/2411.00489/) (2024)
- [Scientific Paper Retrieval with LLM-Guided Semantic-Based Ranking](https://mlsanity.com/paper/2505.21815/) (2025)
- [CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMs](https://mlsanity.com/paper/2501.15067/) (2025)
- [MuaLLM: A Multimodal Large Language Model Agent for Circuit Design Assistance with Hybrid Contextual Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.08137/) (2025)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [Med-R2: Crafting Trustworthy LLM Physicians via Retrieval and Reasoning of Evidence-Based Medicine](https://mlsanity.com/paper/2501.11885/) (2025)
- [IL-PCSR: Legal Corpus for Prior Case and Statute Retrieval](https://mlsanity.com/paper/2511.00268/) (2025)
- [OnRL-RAG: Real-Time Personalized Mental Health Dialogue System](https://mlsanity.com/paper/2504.02894/) (2025)
- [Harnessing Collective Intelligence of LLMs for Robust Biomedical QA: A Multi-Model Approach](https://mlsanity.com/paper/2508.01480/) (2025)
- [AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systems](https://mlsanity.com/paper/2601.11903/) (2026)
- [KohakuRAG: A simple RAG framework with hierarchical document indexing](https://mlsanity.com/paper/2603.07612/) (2026)
- [Rethinking RAG in Long Videos: What to Retrieve and How to Use It?](https://mlsanity.com/paper/2606.13141/) (2026)
- [Ancient Wisdom, Modern Tools: Exploring Retrieval-Augmented LLMs for Ancient Indian Philosophy](https://mlsanity.com/paper/2408.11903/) (2024)
- [KeyKnowledgeRAG (K^2RAG): An Enhanced RAG method for improved LLM question-answering capabilities](https://mlsanity.com/paper/2507.07695/) (2025)
- [Introducing the Swiss Food Knowledge Graph: AI for Context-Aware Nutrition Recommendation](https://mlsanity.com/paper/2507.10156/) (2025)
- [SAGE: Structure Aware Graph Expansion for Retrieval of Heterogeneous Data](https://mlsanity.com/paper/2602.16964/) (2026)

## Abstract (arXiv)

Traditional information retrieval is based on sparse bag-of-words vector representations of documents and queries. More recent deep-learning approaches have used dense embeddings learned using a transformer-based large language model. We show that on a classic benchmark on scientific document retrieval in the medical domain of cystic fibrosis, that both of these models perform roughly equivalently. Notably, dense vectors from the state-of-the-art SPECTER2 model do not significantly enhance performance. However, a hybrid model that we propose combining these methods yields significantly better results, underscoring the merits of integrating classical and contemporary deep learning techniques in information retrieval in the domain of specialized scientific documents.