mlsanityData to 14 Oct 2026

Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Reranking

arXiv 2509.06472 ↗First posted 8 Sep 2025cs.IRcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large Language Models (LLMs) often generate inaccurate responses (hallucinations) when faced with questions beyond their knowledge scope. Retrieval-Augmented Generation (RAG) addresses this by leveraging external knowledge, but a critical challenge remains: determining whether retrieved contexts effectively enhance the model`s ability to answer specific queries. This challenge underscores the importance of knowledge boundary awareness, which current methods-relying on discrete labels or limited signals-fail to address adequately, as they overlook the rich information in LLMs` continuous internal hidden states. To tackle this, we propose a novel post-retrieval knowledge filtering approach. First, we construct a confidence detection model based on LLMs` internal hidden states to quantify how retrieved contexts enhance the model`s confidence. Using this model, we build a preference dataset (NQ_Rerank) to fine-tune a reranker, enabling it to prioritize contexts preferred by the downstream LLM during reranking. Additionally, we introduce Confidence-Based Dynamic Retrieval (CBDR), which adaptively triggers retrieval based on the LLM`s initial confidence in the original question, reducing knowledge conflicts and improving efficiency. Experimental results demonstrate significant improvements in accuracy for context screening and end-to-end RAG performance, along with a notable reduction in retrieval costs while maintaining competitive accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 40 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

28 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation11 Dec
  2. 2024Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training31 May

Other references · 26

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  3. 2023A Survey of Large Language Modelscited by 2,576
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  9. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  10. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
  11. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  12. 2023Do Large Language Models Know What They Don't Know?cited by 243
14 more
  1. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
  2. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 147
  3. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 86
  4. 2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
  5. 2024SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generationcited by 40
  6. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  7. 2024Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluationcited by 38
  8. 2025Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perceptioncited by 27
  9. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 14
  10. 2025Enhancing LLM Reliability via Explicit Knowledge Boundary Modelingcited by 13
  11. 2025Improving Generalization in Intent Detection: GRPO with Reward-Based Curriculum Samplingcited by 12
  12. 2025Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarkscited by 10
  13. 2025LLM-Independent Adaptive RAG: Let the Question Speak for Itselfcited by 7
  14. 2024Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-Confidence Reranker: A Training-Free Approach for Enhancing Retrieval-Augmented Generation Systems
  2. 2024KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models
  3. 2025Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
  4. 2025Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion
  5. 2026Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs
  6. 2024CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control
  7. 2025After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generation
  8. 2024Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMs