# Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrieval

arXiv: https://arxiv.org/abs/2410.13339 · HTML: https://mlsanity.com/paper/2410.13339/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps](https://mlsanity.com/paper/2407.07071/) (2024)
- [End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering](https://mlsanity.com/paper/2106.05346/) (2021)
- [Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory](https://mlsanity.com/paper/2305.02437/) (2023)
- [Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators](https://mlsanity.com/paper/2406.13415/) (2024)

## Built on by (most cited first, 29 of 29)

- [Parametric Retrieval Augmented Generation](https://mlsanity.com/paper/2501.15915/) (2025)
- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond](https://mlsanity.com/paper/2503.20589/) (2025)
- [Stop-RAG: Value-Based Retrieval Control for Iterative RAG](https://mlsanity.com/paper/2510.14337/) (2025)
- [DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router](https://mlsanity.com/paper/2507.22050/) (2025)
- [Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control](https://mlsanity.com/paper/2502.12145/) (2025)
- [MASH: Modeling Abstention via Selective Help-Seeking](https://mlsanity.com/paper/2510.01152/) (2025)
- [LLMs Should Express Uncertainty Explicitly](https://mlsanity.com/paper/2604.05306/) (2026)
- [Conflict-Aware Soft Prompting for Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.15253/) (2025)
- [Skill-RAG: Failure-State-Aware Retrieval Augmentation via Hidden-State Probing and Skill Routing](https://mlsanity.com/paper/2604.15771/) (2026)
- [Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs](https://mlsanity.com/paper/2505.23299/) (2025)
- [Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method](https://mlsanity.com/paper/2508.17862/) (2025)
- [DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search](https://mlsanity.com/paper/2508.20353/) (2025)
- [Lightweight Query Routing for Adaptive RAG: A Baseline Study on RAGRouter-Bench](https://mlsanity.com/paper/2604.03455/) (2026)
- [When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering](https://mlsanity.com/paper/2601.19827/) (2026)
- [Memory Control Signals Emerge Before Action in Long Horizon Agents](https://mlsanity.com/paper/2609.27286/) (2026)
- [Return or Revise? Learning When Revision Helps Retrieval-Augmented QA](https://mlsanity.com/paper/2609.30087/) (2026)
- [Iterative Tree Analysis for Medical Critics](https://mlsanity.com/paper/2501.10642/) (2025)
- [ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection](https://mlsanity.com/paper/2601.15820/) (2026)
- [Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity](https://mlsanity.com/paper/2604.04692/) (2026)
- [RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigm](https://mlsanity.com/paper/2604.20860/) (2026)
- [A Triple-Robustness Analysis of Retrieval-Augmented Generation for Multi-Hop Requirements Traceability](https://mlsanity.com/paper/2608.00705/) (2026)
- [Universal Pathologies, Conditional Consequences: A Triple-Robustness Analysis of RAG for Multi-Hop Traceability](https://mlsanity.com/paper/2608.05153/) (2026)
- [Knowing Before Answering: Decoding Language Models for Reliable RAG](https://mlsanity.com/paper/2608.27661/) (2026)
- [On-Demand Attention: Language Models Know When to Recall](https://mlsanity.com/paper/2609.20734/) (2026)
- [Before Answering: Evidence Sufficiency under Size-Matched Memory Construction](https://mlsanity.com/paper/2609.32269/) (2026)
- [Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG](https://mlsanity.com/paper/2609.37469/) (2026)
- [Locating Answer-Correctness Signals in Frozen Large Language Models](https://mlsanity.com/paper/2609.37700/) (2026)
- [CASE: Cost-Aware Stopping for Efficient Long-Video Agents](https://mlsanity.com/paper/2610.05400/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances language models by retrieving and incorporating relevant external knowledge. However, traditional retrieve-and-generate processes may not be optimized for real-world scenarios, where queries might require multiple retrieval steps or none at all. In this paper, we propose a Probing-RAG, which utilizes the hidden state representations from the intermediate layers of language models to adaptively determine the necessity of additional retrievals for a given query. By employing a pre-trained prober, Probing-RAG effectively captures the model's internal cognition, enabling reliable decision-making about retrieving external documents. Experimental results across five open-domain QA datasets demonstrate that Probing-RAG outperforms previous methods while reducing the number of redundant retrieval steps.