mlsanityData to 14 Oct 2026

MedHal-Loc: Are "Explainable-by-Architecture" Medical Hallucination Detectors Faithful Localizers? A Localization Benchmark

arXiv 2606.21517 ↗First posted 19 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Detecting hallucinations in clinical text is increasingly framed as an explainability problem: systems should not merely flag an unreliable response but point to the offending span. Architectures built around knowledge-graph (KG) triple decomposition are marketed for exactly this auditability, yet their localization ability is typically assumed rather than measured. We introduce MedHal-Loc, a benchmark and metric for localization faithfulness -- whether a detector's top-ranked error unit actually overlaps the erroneous span. The controlled subset comprises 300 PubMedQA-derived statements with single, span-level errors injected across four localizable types (entity substitution, relation error, mechanism misattribution, invention), yielding gold spans by construction; a complementary natural subset documents that real hallucinations are dominated by diffuse conclusion-flips that resist span localization (a human expert accepted 1/18 candidate spans). Evaluating four fine-grained paradigms, we find that NLI-per-clause, consistency-per-sentence, and the dedicated span detector FAVA all localize well above chance, whereas an elaborate KG-triple pipeline localizes no better than chance (+3.3pp, n.s.), bottlenecked by ~59% entity-extraction coverage -- despite competitive detection F1 (0.609). Detection competence does not imply faithful localization; architectural explainability must be validated, not presumed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 24 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

19 of 24 references.

References · 19

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  11. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  12. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
7 more
  1. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 237
  2. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 157
  3. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 149
  4. 2024GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Frameworkcited by 30
  5. 2025Hallucination Detection in Large Language Models with Metamorphic Relationscited by 24
  6. 2024A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagationcited by 21
  7. 2024Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modelingcited by 9

Built on by

Not cited yet.