mlsanityData to 7 Oct 2026

Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint

arXiv 2402.11893 ↗First posted 19 Feb 2024cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

Large language models internalize enormous parametric knowledge during pre-training. Concurrently, realistic applications necessitate external contextual knowledge to aid models on the underlying tasks. This raises a crucial dilemma known as knowledge conflicts, where the contextual knowledge clashes with the However, existing decoding works are specialized in resolving knowledge conflicts and could inadvertently deteriorate performance in absence of conflicts. In this paper, we propose an adaptive decoding method, termed as contextual information-entropy constraint decoding (COIECD), to discern whether the knowledge conflicts occur and resolve them. It can improve the model's faithfulness to conflicting context, and simultaneously maintain high performance among non- Our experiments show that COIECD exhibits strong performance and robustness over knowledge conflicts in realistic datasets. Code is available.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

35 of 46 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  12. 2019Language Models as Knowledge Bases?cited by 1,922
23 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  2. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  3. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  4. 2018Hierarchical Neural Story Generationcited by 1,182
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  6. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  11. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  12. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  13. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  14. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  15. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  16. 2022RealTime QA: What's the Answer Right Now?cited by 239
  17. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  18. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  19. 2022Large Language Models with Controllable Working Memorycited by 169
  20. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  21. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  22. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 103
  23. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 93

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 17; 2026: 8.

27 papers.

Sort:

By area:

  1. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
  2. 2024AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledgecited by 35
  3. 2025Taming Knowledge Conflicts in Language Modelscited by 27
  4. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 27
  5. 2025ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generationcited by 11
  6. 2024Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliancecited by 9
  7. 2025Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Modelscited by 9
  8. 2025CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Modelscited by 9
  9. 2025Can LLMs Reconcile Knowledge Conflicts in Counterfactual Reasoningcited by 7
  10. 2025Stochastic Chameleons: Irrelevant Context Hallucinations Reveal Class-Based (Mis)Generalization in LLMscited by 5
  11. 2025Conflict-Aware Soft Prompting for Retrieval-Augmented Generationcited by 5
  12. 2025Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wildcited by 4
15 more
  1. 2025Continuously Steering LLMs Sensitivity to Contextual Knowledge with Proxy Modelscited by 4
  2. 2025CUB: Benchmarking Context Utilisation Techniques for Language Modelscited by 3
  3. 2025Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Datacited by 3
  4. 2026CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answeringcited by 2
  5. 2026Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decodingcited by 2
  6. 2025Exploiting Contextual Knowledge in LLMs through V-usable Information based Layer Enhancementcited by 1
  7. 2025Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoningcited by 1
  8. 2026When RAG Hurts: Diagnosing and Mitigating Attention Distraction in Retrieval-Augmented LVLMscited by 1
  9. 2026Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decodingcited by 1
  10. 2026From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMscited by 1
  11. 2025UniKnow: A Unified Framework for Reliable Language Model Behavior across Parametric and External Knowledgecited by 0
  12. 2025HAVE: Head-Adaptive Gating and ValuE Calibration for Hallucination Mitigation in Large Language Modelscited by 0
  13. 2026Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAGcited by 0
  14. 2026FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAGcited by 0
  15. 2026Regime Boundary Alignment for Evidence-Gated Question Answeringcited by 0