mlsanityData to 7 Oct 2026

Inference Scaling for Bridging Retrieval and Augmented Generation

arXiv 2412.10684 ↗First posted 14 Dec 2024cs.CLcited by 7

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation (RAG) has emerged as a popular approach to steering the output of a large language model (LLM) by incorporating retrieved contexts as inputs. However, existing work observed the generator bias, such that improving the retrieval results may negatively affect the outcome. In this work, we show such bias can be mitigated, from inference scaling, aggregating inference calls from the permuted order of retrieved contexts. The proposed Mixture-of-Intervention (MOI) explicitly models the debiased utility of each passage with multiple forward passes to construct a new ranking. We also show that MOI can leverage the retriever's prior knowledge to reduce the computational cost by minimizing the number of permutations considered and lowering the cost per LLM call. We showcase the effectiveness of MOI on diverse RAG tasks, improving ROUGE-L on MS MARCO and EM on HotpotQA benchmarks by ~7 points.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 30 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

21 of 30 references.

References · 21

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  6. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  8. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  9. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  10. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  11. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  12. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
9 more
  1. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  2. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  3. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  4. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  5. 2024Eliminating Position Bias of Language Models: A Mechanistic Approachcited by 61
  6. 2024FIRST: Faster Improved Listwise Reranking with Single Token Decodingcited by 54
  7. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  8. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 31
  9. 2023Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deteriorationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  2. 2026Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibrationcited by 4
  3. 2025RoToR: Towards More Reliable Responses for Order-Invariant Inputscited by 2
  4. 2025Relevance to Utility: Process-Supervised Rewrite for RAGcited by 1
  5. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systemscited by 0
  6. 2026The Laws of Context Allocation: Causal Measurement and Closed-Loop Orchestration in Generative Searchcited by 0
  7. 2026Assessing the Downstream Utility of Evidence-Aware Retrieval in RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Inference Scaling for Long-Context Retrieval Augmented Generation
  2. 2024CARROT: A Learned Cost-Constrained Retrieval Optimization System for RAG
  3. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  4. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  5. 2026From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation
  6. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  7. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  8. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting