mlsanityData to 7 Oct 2026

Trusting Your Evidence: Hallucinate Less with Context-aware Decoding

arXiv 2305.14739 ↗First posted 24 May 2023cs.CLcited by 319

Not on the map. Why?

Cited by 45 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language models (LMs) often struggle to pay enough attention to the input context, and generate texts that are unfaithful or contain hallucinations. To mitigate this issue, we present context-aware decoding (CAD), which follows a contrastive output distribution that amplifies the difference between the output probabilities when a model is used with and without context. Our experiments show that CAD, without additional training, significantly improves the faithfulness of different LM families, including OPT, GPT, LLaMA and FLAN-T5 for summarization tasks (e.g., 14.3% gain for LLaMA in factuality metrics). Furthermore, CAD is particularly effective in overriding a model's prior knowledge when it contradicts the provided context, leading to substantial improvements in tasks where resolving the knowledge conflict is essential.

Key facts. Cited by 319 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

36 of 46 references.

References · 36

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  11. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  12. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
24 more
  1. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  2. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  3. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  4. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  5. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  6. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 303
  7. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  8. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  9. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  10. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  11. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  12. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  13. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  14. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  15. 2023Context-faithful Prompting for Large Language Modelscited by 76
  16. 2020Improved Natural Language Generation via Loss Truncationcited by 75
  17. 2022Mutual Information Alleviates Hallucinations in Abstractive Summarizationcited by 67
  18. 2020Multi-Fact Correction in Abstractive Text Summarizationcited by 66
  19. 2021Improving Factual Consistency of Abstractive Summarization via Question Answeringcited by 55
  20. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  21. 2022Nonparametric Masked Language Modelingcited by 39
  22. 2021Coherence boosting: When your pretrained language model is not paying enough attentioncited by 38
  23. 2022Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Searchcited by 23
  24. 2022Improved Beam Search for Hallucination Mitigation in Abstractive Summarizationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 111; 2025: 93; 2026: 89.

319 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  4. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  5. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  6. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  7. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  8. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  9. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 196
  10. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
  11. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  12. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
38 more
  1. 2024Language Models with Conformal Factuality Guaranteescited by 128
  2. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  3. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Modelscited by 93
  4. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  5. 2023Stay on topic with Classifier-Free Guidancecited by 90
  6. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  7. 2024Tuning Language Models by Proxycited by 88
  8. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  9. 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
  10. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  11. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  12. 2023In-context Pretraining: Language Modeling Beyond Document Boundariescited by 76
  13. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  14. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Modelscited by 72
  15. 2024Factuality of Large Language Models: A Surveycited by 64
  16. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 64
  17. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  18. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  19. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
  20. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steeringcited by 62
  21. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  22. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  23. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 50
  24. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  25. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  26. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49
  27. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  28. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  29. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  30. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  31. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  32. 2024Evaluating Copyright Takedown Methods for Language Modelscited by 42
  33. 2023Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucinationcited by 41
  34. 2024Distract Large Language Models for Automatic Jailbreak Attackcited by 40
  35. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  36. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
  37. 2024Reducing Tool Hallucination via Reliability Alignmentcited by 37
  38. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 36

50 most cited of 319 All 319 (JSON)