mlsanityData to 7 Oct 2026

Context-DPO: Aligning Language Models for Context-Faithfulness

arXiv 2412.15280 ↗First posted 18 Dec 2024cs.CL cs.AI cs.IRcited by 50

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intentions and values, improving context-faithfulness through alignment remains underexplored. To address this, we propose Context-DPO, the first alignment method specifically designed to enhance LLMs' context-faithfulness. We introduce ConFiQA, a benchmark that simulates Retrieval-Augmented Generation (RAG) scenarios with knowledge conflicts to evaluate context-faithfulness. By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization. Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35% to 280% improvements on popular open-source models. Further analysis demonstrates that Context-DPO preserves LLMs' generative capabilities while providing interpretable insights into context utilization. Our code and data are released at https://github.com/byronBBL/Context-DPO

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 54 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

39 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Large Language Model Alignment: A Survey26 Sep

Other references · 38

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  9. 2022Mass-Editing Memory in a Transformercited by 976
  10. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  11. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  12. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
26 more
  1. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  2. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  3. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  4. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  5. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  6. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  7. 2022Prompting GPT-3 To Be Reliablecited by 258
  8. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  9. 2020Modifying Memories in Transformer Modelscited by 222
  10. 2023Fine-tuning Language Models for Factualitycited by 219
  11. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  12. 2020How Context Affects Language Models' Factual Predictionscited by 181
  13. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  14. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  15. 2022Training Language Models with Memory Augmentationcited by 112
  16. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  17. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  18. 2024GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solvingcited by 44
  19. 2024KG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniquescited by 37
  20. 2024Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Factscited by 36
  21. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  22. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 23
  23. 2024Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulnesscited by 17
  24. 2024CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Modelscited by 15
  25. 2024Adaptive Token Biaser: Knowledge Editing via Biasing Key Entitiescited by 14
  26. 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Stylecited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 28; 2026: 22.

50 papers.

Sort:

By area:

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  2. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
  3. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 27
  4. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinkingcited by 19
  5. 2025a1: Steep Test-time Scaling Law via Environment Augmented Generationcited by 13
  6. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 13
  7. 2025Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMscited by 12
  8. 2025ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generationcited by 11
  9. 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcementcited by 11
  10. 2025SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimizationcited by 10
  11. 2025Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoningcited by 8
  12. 2025Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolutioncited by 7
38 more
  1. 2025Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learningcited by 7
  2. 2025Large Language Models as Computable Approximations to Solomonoff Inductioncited by 6
  3. 2025Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarizationcited by 6
  4. 2025MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generationcited by 6
  5. 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generationcited by 6
  6. 2026What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generationcited by 6
  7. 2025ConSens: Assessing context grounding in open-book question answeringcited by 4
  8. 2025Alignment with Fill-In-the-Middle for Enhancing Code Generationcited by 4
  9. 2026EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Modelscited by 4
  10. 2025Injecting External Knowledge into the Reasoning Process Enhances Retrieval-Augmented Generationcited by 3
  11. 2025FaithLens: Detecting and Explaining Faithfulness Hallucinationcited by 3
  12. 2026ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Modelscited by 3
  13. 2026SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generationcited by 3
  14. 2025Conflicts in Texts: Data, Implications and Challengescited by 2
  15. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  16. 2025Noise Augmented Fine Tuning for Mitigating Hallucinations in Large Language Modelscited by 1
  17. 2025Exploiting Contextual Knowledge in LLMs through V-usable Information based Layer Enhancementcited by 1
  18. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 1
  19. 2025Understanding Parametric Knowledge Injection in Retrieval-Augmented Generationcited by 1
  20. 2026Trust or Abstain? A Self-Aware RAG Approachcited by 1
  21. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1
  22. 2026Does a Tool Result Carry More Authority Than Plain Text? Three Prospective Studies of False-Claim Adoption in a Synthetic Assignment Task with Claude Opus 5cited by 1
  23. 2025RACE-Align: Retrieval-Augmented and Chain-of-Thought Enhanced Preference Alignment for Large Language Modelscited by 0
  24. 2025SSFO: Self-Supervised Faithfulness Optimization for Retrieval-Augmented Generationcited by 0
  25. 2026PromptCD: Test-Time Behavior Enhancement via Polarity-Prompt Contrastive Decodingcited by 0
  26. 2026FAITH: Factuality Alignment through Integrating Trustworthiness and Honestnesscited by 0
  27. 2026IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Toolscited by 0
  28. 2026OCC-RAG: Optimal Cognitive Core for Faithful Question Answeringcited by 0
  29. 2026BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigationcited by 0
  30. 2026PURPOSE: Poisoning Conflict Resolution in RAG via Proxy-Fact-Grounded Updatescited by 0
  31. 2026When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generationcited by 0
  32. 2026GRIP: Grounded Reasoning via Information-Restricted Premisescited by 0
  33. 2026MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Modelscited by 0
  34. 2026Co-Evolving Structured Knowledge and Reasoning in Language Modelscited by 0
  35. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  36. 2026The Answer Path and the Grounding Instruction in LLM Question Answering over Knowledge Graphscited by 0
  37. 2026Parameters vs. Context: TRACE Fine-Tuning for Robust Retrieval-Augmented Generationcited by 0
  38. 2026Direct Hidden-State Alignment: Mapping and Controlling Preference Expression in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity
  2. 2025Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization
  3. 2026Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective
  4. 2025LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
  5. 2024Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks
  6. 2026Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation
  7. 2024Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?
  8. 2025SPRI: Aligning Large Language Models with Context-Situated Principles