mlsanityData to 7 Oct 2026

FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"

arXiv 2410.03727 ↗First posted 30 Sep 2024cs.CL cs.AI cs.LGcited by 85

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Ensuring faithfulness to context in large language models (LLMs) and retrieval-augmented generation (RAG) systems is crucial for reliable deployment in real-world applications, as incorrect or unsupported information can erode user trust. Despite advancements on standard benchmarks, faithfulness hallucination-where models generate responses misaligned with the provided context-remains a significant challenge. In this work, we introduce FaithEval, a novel and comprehensive benchmark tailored to evaluate the faithfulness of LLMs in contextual scenarios across three diverse tasks: unanswerable, inconsistent, and counterfactual contexts. These tasks simulate real-world challenges where retrieval mechanisms may surface incomplete, contradictory, or fabricated information. FaithEval comprises 4.9K high-quality problems in total, validated through a rigorous four-stage context construction and validation framework, employing both LLM-based auto-evaluation and human validation. Our extensive study across a wide range of open-source and proprietary models reveals that even state-of-the-art models often struggle to remain faithful to the given context, and that larger models do not necessarily exhibit improved faithfulness.Project is available at: https://github.com/SalesforceAIResearch/FaithEval.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 64 of its 74 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

64 of 74 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence16 Apr
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct
  3. 2023Benchmarking Large Language Models in Retrieval-Augmented Generation4 Sep

Other references · 61

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Mistral 7Bcited by 3,065
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2023A Survey of Large Language Modelscited by 2,537
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
38 more
  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  6. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  7. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  8. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  9. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  11. 2023A Survey on Large Language Models for Recommendationcited by 378
  12. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  13. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  14. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  15. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  16. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  17. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
  18. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  19. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  20. 2023Do Large Language Models Know What They Don't Know?cited by 238
  21. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  22. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  23. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  24. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  25. 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
  26. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  27. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  28. 2024Long-form factuality in large language modelscited by 146
  29. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  30. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 142
  31. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  32. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  33. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  34. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  35. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  36. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  37. 2022SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentationcited by 82
  38. 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 77

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 34; 2026: 47.

85 papers.

Sort:

By area:

  1. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  2. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 65
  3. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 60
  4. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  5. 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 44
  6. 2025MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Modelscited by 33
  7. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 27
  8. 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contextscited by 25
  9. 2025Steering MoE LLMs via Expert (De)Activationcited by 23
  10. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  11. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMscited by 21
  12. 2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboardscited by 19
38 more
  1. 2025Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLMcited by 16
  2. 2025ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generationcited by 11
  3. 2025Feedback Friction: LLMs Struggle to Fully Incorporate External Feedbackcited by 11
  4. 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?cited by 10
  5. 2025FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Financecited by 8
  6. 2025RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Modelscited by 8
  7. 2025Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasetscited by 7
  8. 2025Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learningcited by 7
  9. 2025Faithful-First Reasoning, Planning, and Acting for Multimodal LLMscited by 7
  10. 2026SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directionscited by 7
  11. 2025GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMscited by 6
  12. 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generationcited by 6
  13. 2024The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Modelscited by 5
  14. 2025Consensus or Conflict? Fine-Grained Evaluation of Conflicting Answers in Question-Answeringcited by 5
  15. 2025Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Taskscited by 5
  16. 2024Unanswerability Evaluation for Retrieval Augmented Generationcited by 4
  17. 2025Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citationcited by 4
  18. 2025aiXamine: Simplified LLM Safety and Securitycited by 4
  19. 2025Knowing What's Missing: Assessing Information Sufficiency in Question Answeringcited by 3
  20. 2025Do Retrieval-Augmented Language Models Adapt to Varying User Needs?cited by 2
  21. 2025Beyond Facts: Evaluating Intent Hallucination in Large Language Modelscited by 2
  22. 2025Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generationcited by 2
  23. 2026CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answeringcited by 2
  24. 2026Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluatorscited by 2
  25. 2025Efficient OpAmp Adaptation for Zoom Attention to Golden Contextscited by 1
  26. 2025Fine-Grained Detection of Context-Grounded Hallucinations Using LLMscited by 1
  27. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 1
  28. 2026Advancing ESG Intelligence: An Expert-level Agent and Comprehensive Benchmark for Sustainable Financecited by 1
  29. 2026Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QAcited by 1
  30. 2026CTRL-RAG: Contrastive Likelihood Reward Based Reinforcement Learning for Context-Faithful RAG Modelscited by 1
  31. 2026A Benchmark Construction and Evaluation Framework for Specialist Domains: Case Study on Defense-related Documentscited by 1
  32. 2026Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decodingcited by 1
  33. 2026Faithfulness-QA: A Counterfactual Entity Substitution Dataset for Training Context-Faithful RAG Modelscited by 1
  34. 2026LLM-Oriented Information Retrieval: A Denoising-First Perspectivecited by 1
  35. 2026Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checkingcited by 1
  36. 2026Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluationcited by 1
  37. 2026Information Discernment in Large Language Modelscited by 1
  38. 2026Learning When to Trust via Selective Context Preference Optimizationcited by 1

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective
  2. 2025Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization
  3. 2023RELIC: Investigating Large Language Model Responses using Self-Consistency
  4. 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
  5. 2024Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity
  6. 2025Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval Augmented Generation
  7. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Survey
  8. 2024A review of faithfulness metrics for hallucination assessment in Large Language Models