# Pre-retrieval Query Clustering for Adaptive Top-k Document Retrieval in RAG Systems

arXiv: https://arxiv.org/abs/2609.13489 · HTML: https://mlsanity.com/paper/2609.13489/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 11 Sep 2026.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 7 of 7)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [A Theoretical Analysis of NDCG Type Ranking Measures](https://mlsanity.com/paper/1304.6480/) (2013)
- [HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applications](https://mlsanity.com/paper/2409.09046/) (2024)
- [Experiments on Generalizability of BERTopic on Multi-Domain Short Text](https://mlsanity.com/paper/2212.08459/) (2022)
- [Query-specific Variable Depth Pooling via Query Performance Prediction towards Reducing Relevance Assessment Effort](https://mlsanity.com/paper/2304.11752/) (2023)
- [Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applications](https://mlsanity.com/paper/2511.14769/) (2025)

## Abstract (arXiv)

RAG systems commonly retrieve a fixed number of documents (top-k) to ground generation, but this static approach is brittle: simple queries suffer over-retrieval (adding noise and cost) while complex queries are under-retrieved, causing recall failures that cascade into incorrect answers. Motivated by the question of how many documents must be retrieved to answer an arbitrary query reliably, we propose a practical, general framework for query-adaptive retrieval depth. Offline, we estimate per-query retrieval difficulty by measuring NDCG under the default retriever and deriving a query-specific saturation point k* from the NDCG-k curve. Because computing these signals online is expensive, we cluster a large set of queries in embedding space and summarize each cluster with a recommended retrieval depth that targets high coverage (e.g., ~95%) using a mean-plus-variance rule. At runtime, the system assigns an incoming query to a cluster and selects the corresponding top-k in constant time. Compared with post-retrieval confidence methods that rely on clustering retrieved documents, our approach is pre-retrieval and query-centric, making it robust in heterogeneous, case-like corpora and applicable across domains such as legal, healthcare, finance, and enterprise search. Finally, this framework has been tested in full-traffic queries that improved F1 by over 36% while reducing token usage by 14% on low-complexity clusters without accuracy loss.