mlsanityData to 7 Oct 2026

RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Models

arXiv 2405.14486 ↗First posted 23 May 2024cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown impressive capabilities but also a concerning tendency to hallucinate. This paper presents RefChecker, a framework that introduces claim-triplets to represent claims in LLM responses, aiming to detect fine-grained hallucinations. In RefChecker, an extractor generates claim-triplets from a response, which are then evaluated by a checker against a reference. We delineate three task settings: Zero, Noisy and Accurate Context, to reflect various real-world use cases. We curated a benchmark spanning various NLP tasks and annotated 11k claim-triplets from 2.1k responses by seven LLMs. RefChecker supports both proprietary and open-source models as the extractor and checker. Experiments demonstrate that claim-triplets enable superior hallucination detection, compared to other granularities such as response, sentence and sub-sentence level claims. RefChecker outperforms prior methods by 6.8 to 26.1 points on our benchmark and the checking results of RefChecker are strongly aligned with human judgments. This work is open sourced at https://github.com/amazon-science/RefChecker

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

18 of 30 references.

References · 18

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  7. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  8. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  9. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  10. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  11. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  12. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
6 more
  1. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 128
  2. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  3. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  4. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  5. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  6. 2023A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detectioncited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 9; 2026: 9.

24 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 100
  3. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  4. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  5. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  6. 2024Logical Consistency of Large Language Models in Fact-checkingcited by 15
  7. 2026Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agentscited by 10
  8. 2025UQLM: A Python Package for Uncertainty Quantification in Large Language Modelscited by 9
  9. 2024The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal Statescited by 8
  10. 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signalscited by 8
  11. 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 6
  12. 2024Enabling Scalable Evaluation of Bias Patterns in Medical LLMscited by 5
12 more
  1. 2025Improving Context Fidelity via Native Retrieval-Augmented Reasoningcited by 5
  2. 2025Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspectivecited by 3
  3. 2025MultiRAG: A Knowledge-guided Framework for Mitigating Hallucination in Multi-source Retrieval Augmented Generationcited by 3
  4. 2026PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinationscited by 3
  5. 2026Fine-grained Claim-level RAG Benchmark for Lawcited by 2
  6. 2026FinReflectKG -- HalluBench: GraphRAG Hallucination Benchmark for Financial Question Answering Systemscited by 1
  7. 2025Iterative Tree Analysis for Medical Criticscited by 0
  8. 2026Probabilistic Guarantees for Reducing Contextual Hallucinations in LLMscited by 0
  9. 2026From Retinal Evidence to Safe Decisions: RETINA-SAFE and ECRT for Hallucination Risk Triage in Medical LLMscited by 0
  10. 2026Graph Alignment Topology as an Inductive Bias for Grounding Detectioncited by 0
  11. 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinationscited by 0
  12. 2026How Much Do Legal RAG Systems Still Hallucinate?cited by 0