mlsanityData to 7 Oct 2026

Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts

arXiv 2408.01084 ↗First posted 2 Aug 2024cs.CLcited by 19

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

When using large language models (LLMs) in knowledge-intensive tasks, such as open-domain question answering, external context can bridge the gap between external knowledge and the LLMs' parametric knowledge. Recent research has been developed to amplify contextual knowledge over the parametric knowledge of LLMs with contrastive decoding approaches. While these approaches could yield truthful responses when relevant context is provided, they are prone to vulnerabilities when faced with noisy contexts. We extend the scope of previous studies to encompass noisy contexts and propose adaptive contrastive decoding (ACD) to leverage contextual influence effectively. ACD demonstrates improvements in open-domain question answering tasks compared to baselines, especially in robustness by remaining undistracted by noisy contexts in retrieval-augmented generation.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 45 references that are in the map. First posted on arXiv 2 Aug 2024.

Builds on

23 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  7. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
10 more
  1. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  2. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  3. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  5. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  6. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  7. 2023In-context Pretraining: Language Modeling Beyond Document Boundariescited by 76
  8. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49
  9. 2021Coherence boosting: When your pretrained language model is not paying enough attentioncited by 38
  10. 2024Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contextscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 11.

19 papers.

Sort:

By area:

  1. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  2. 2025MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation Systemcited by 19
  3. 2025Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learningcited by 18
  4. 2024Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perceptioncited by 17
  5. 2024When to Speak, When to Abstain: Contrastive Decoding with Abstentioncited by 7
  6. 2025CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generationcited by 6
  7. 2025CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergycited by 4
  8. 2025CUB: Benchmarking Context Utilisation Techniques for Language Modelscited by 3
  9. 2026Steer Model beyond Assistant: Controlling System Prompt Strength via Contrastive Decodingcited by 1
  10. 2026Evaluating Chunking Strategies For Retrieval-Augmented Generation in Oil and Gas Enterprise Documentscited by 1
  11. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1
  12. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0
7 more
  1. 2026Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generationcited by 0
  2. 2026Attention-Guided Layer Selection for Contrastive Decoding in Large Language Modelscited by 0
  3. 2026Decoupled Contrastive Decoding via Expert-Aligned Draftingcited by 0
  4. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0
  5. 2026Training-Free VLM Personalization via Calibrated Residual Decodingcited by 0
  6. 2026Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognitioncited by 0
  7. 2026Regime Boundary Alignment for Evidence-Gated Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answering
  2. 2026Interpretable Uncertainty for Adaptive Retrieval and Reasoning in Question Answering
  3. 2025SUGAR: Leveraging Contextual Confidence for Smarter Retrieval
  4. 2025Retrieval-augmented Decoding for Improving Truthfulness in Open-ended Generation
  5. 2024Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts
  6. 2024RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation
  7. 2025Improving Context Fidelity via Native Retrieval-Augmented Reasoning
  8. 2024Assessing "Implicit" Retrieval Robustness of Large Language Models