mlsanityData to 14 Oct 2026

Re-ranking and Late Interaction Drive Retrieval Quality: A Controlled Comparison of RAG Strategies for Scientific Question Answering

arXiv 2609.38473 ↗First posted 29 Sep 2026cs.IR cs.AI cs.LGcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval also fits: Generation and context integration

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) is now the standard way to ground Large Language Models (LLMs) in external knowledge, yet the design space of retrieval pipelines is large and the trade-offs between variants are not well understood, especially on domain-specific corpora at realistic scale. In this work, we present a controlled comparison of six retrieval strategies for scientific question answering: (i) classic top-k dense retrieval, (ii) LLM-based query rephrasing, (iii) query rephrasing followed by LLM-based reranking, (iv) multi-query fusion via Reciprocal Rank Fusion (RRF), (v) an agentic tool-call pipeline in which the generator decides for itself whether to retrieve, and (vi) late-interaction retrieval with ColBERTv2. All six pipelines share the same generator (Meta-Llama/Llama-3.1-8B-Instruct), prompt, and evaluation protocol; the five single-vector pipelines additionally share SPECTER2 embeddings and a Chroma vector store; and all six retrieve from the full corpus of 463,971 arXiv papers dated 2024-2025. To support reproducible, large-scale evaluation, we also release a synthetic question dataset of 19,484 problem-statement and methodology questions generated by Llama-3.1-8B-Instruct from a random sample of 10,000 papers across academic domains (query generation succeeded for 9,742 of them), and every strategy is evaluated on this same query set. We describe the architecture and implementation of each pipeline, release the code and the synthetic question dataset, and evaluate each strategy with an LLM-as-a-judge protocol along multiple quality dimensions, together with direct gold-paper retrieval metrics. The result is an open testbed for studying the cost and quality trade-offs of RAG design choices on a research-literature corpus, and a basis for future work on faithfulness, retrieval robustness, and agentic retrieval.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 28 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

23 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation31 Mar
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  10. 2019Passage Re-ranking with BERTcited by 645
  11. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  12. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
9 more
  1. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  2. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  3. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
  4. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  5. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
  6. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 129
  7. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 101
  8. 2024DMQR-RAG: Diverse Multi-Query Rewriting for RAGcited by 27
  9. 2025MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoningcited by 26

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG
  2. 2025SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generated LLMs
  3. 2026Bridging Static and Agentic RAG for Taiwanese Historical Question Answering
  4. 2026VecTree-RAG: An Agentic Retrieval-Augmented Generation Framework Combining Vector and Tree Retrieval for Efficiency and Accuracy
  5. 2025Question Decomposition for Retrieval-Augmented Generation
  6. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  7. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  8. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems