# Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts

arXiv: https://arxiv.org/abs/2408.01084 · HTML: https://mlsanity.com/paper/2408.01084/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 23 of 23)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity](https://mlsanity.com/paper/2305.13169/) (2023)
- [In-context Pretraining: Language Modeling Beyond Document Boundaries](https://mlsanity.com/paper/2310.10638/) (2023)
- [Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding](https://mlsanity.com/paper/2405.02750/) (2024)
- [Coherence boosting: When your pretrained language model is not paying enough attention](https://mlsanity.com/paper/2110.08294/) (2021)
- [Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contexts](https://mlsanity.com/paper/2403.07556/) (2024)

## Built on by (most cited first, 19 of 19)

- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System](https://mlsanity.com/paper/2503.09600/) (2025)
- [Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learning](https://mlsanity.com/paper/2505.17097/) (2025)
- [Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception](https://mlsanity.com/paper/2410.12788/) (2024)
- [When to Speak, When to Abstain: Contrastive Decoding with Abstention](https://mlsanity.com/paper/2412.12527/) (2024)
- [CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generation](https://mlsanity.com/paper/2508.05534/) (2025)
- [CUB: Benchmarking Context Utilisation Techniques for Language Models](https://mlsanity.com/paper/2505.16518/) (2025)
- [CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergy](https://mlsanity.com/paper/2508.01696/) (2025)
- [Steer Model beyond Assistant: Controlling System Prompt Strength via Contrastive Decoding](https://mlsanity.com/paper/2601.06403/) (2026)
- [Evaluating Chunking Strategies For Retrieval-Augmented Generation in Oil and Gas Enterprise Documents](https://mlsanity.com/paper/2603.24556/) (2026)
- [Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference](https://mlsanity.com/paper/2606.20245/) (2026)
- [Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion](https://mlsanity.com/paper/2604.18106/) (2026)
- [Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.00570/) (2026)
- [Attention-Guided Layer Selection for Contrastive Decoding in Large Language Models](https://mlsanity.com/paper/2607.23067/) (2026)
- [Decoupled Contrastive Decoding via Expert-Aligned Drafting](https://mlsanity.com/paper/2608.12913/) (2026)
- [Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models](https://mlsanity.com/paper/2608.14797/) (2026)
- [Training-Free VLM Personalization via Calibrated Residual Decoding](https://mlsanity.com/paper/2608.22263/) (2026)
- [Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognition](https://mlsanity.com/paper/2608.26213/) (2026)
- [Regime Boundary Alignment for Evidence-Gated Question Answering](https://mlsanity.com/paper/2609.37491/) (2026)

## Abstract (arXiv)

When using large language models (LLMs) in knowledge-intensive tasks, such as open-domain question answering, external context can bridge the gap between external knowledge and the LLMs' parametric knowledge. Recent research has been developed to amplify contextual knowledge over the parametric knowledge of LLMs with contrastive decoding approaches. While these approaches could yield truthful responses when relevant context is provided, they are prone to vulnerabilities when faced with noisy contexts. We extend the scope of previous studies to encompass noisy contexts and propose adaptive contrastive decoding (ACD) to leverage contextual influence effectively. ACD demonstrates improvements in open-domain question answering tasks compared to baselines, especially in robustness by remaining undistracted by noisy contexts in retrieval-augmented generation.