mlsanityData to 14 Oct 2026

Better Together: Complementary Query Rewriting Under a Strong RAG Baseline

arXiv 2609.05637 ↗First posted 4 Sep 2026cs.CL cs.IRcited by 0

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

A popular way to improve Retrieval-Augmented Generation (RAG) is to rewrite the user's question into several variants and search with all of them. We test whether this actually helps once the underlying search is already strong. Under one fixed, competitive pipeline (BGE dense retrieval, cross-encoder reranking, and MMR diversification), we compare four query-rewriting strategies (S1-S4) against two strong LLM baselines (HyDE, Query2Doc) on three datasets (HotpotQA, AmbigNQ, and the 512K-document EnterpriseRAG-Bench) over three seeds with paired-bootstrap significance tests. Our headline result is that rewriting alone is at best competitive with a strong baseline, but combining methods yields outsized gains because different strategies fail on different questions. A post-hoc union of four methods (S1+S3+S4+HyDE) improves HIT@10 over the baseline by +12.5 points on enterprise data (51.70 vs 39.22), and a five-method union reaches 52.98 (+13.8). Budget-matched controls capture only ~40% of this gain, confirming that complementarity, not retrieval budget, is the primary driver. On HotpotQA the union adds +1.6 to +1.8 points (p<0.001), saturating the all-method oracle; on AmbigNQ the same fusion hurts (-2.4 below the best solo, p<0.001), and we analyze when and why. Because rewriting is expensive, we evaluate in simulation a confidence-gated router that runs rewriting only when the baseline's own top-1 score is low. It captures about half of the enterprise full-merge gain (+4.3 HIT@10) while paying rewriting cost on <40% of queries, and automatically declines to rewrite on AmbigNQ. A downstream answer-quality evaluation confirms the router improves F1 by +1.92 (p<0.01) at roughly 40% of the expansion cost. In short: treat query rewriting as a complementary coverage source, applied through cost-aware routing, not as a standalone replacement for a strong baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 8 of its 21 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

8 of 21 references.

References · 8

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  2. 2019Passage Re-ranking with BERTcited by 645
  3. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  4. 2019Document Expansion by Query Predictioncited by 231
  5. 2023Query2doc: Query Expansion with Large Language Modelscited by 149
  6. 2024DMQR-RAG: Diverse Multi-Query Rewriting for RAGcited by 27
  7. 2025LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searcherscited by 11
  8. 2026EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledgecited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG
  2. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAG
  3. 2025RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
  4. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  5. 2026Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?
  6. 2026Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAG
  7. 2026Not All Queries Need Rewriting: When Prompt-Only LLM Refinement Helps and Hurts Dense Retrieval
  8. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control