mlsanityData to 7 Oct 2026

HalluScan: A Systematic Benchmark for Detecting and Mitigating Hallucinations in Instruction-Following LLMs

arXiv 2605.02443 ↗First posted 4 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks, yet they remain susceptible to hallucinations -- generating content that is factually incorrect, unfaithful to provided context, or misaligned with user instructions. We present HalluScan, a comprehensive benchmark framework that systematically evaluates hallucination detection and mitigation across 72 configurations spanning 6 detection methods, 4 open-weight model families, and 3 diverse domains. We introduce three key contributions: (1) HalluScore, a novel composite metric that achieves a Pearson correlation of r = 0.41 with human expert judgments; (2) Adaptive Detection Routing (ADR), an intelligent routing algorithm achieving 2.0x cost reduction with only 0.1% AUROC degradation; and (3) systematic error cascade decomposition revealing substantial variation in hallucination error types across domains. Our experiments reveal that NLI Verification achieves the highest overall AUROC of 0.88, while RAV achieves the second-highest AUROC of 0.66.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

47 of 62 references.

References · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
35 more
  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  6. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  7. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  8. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  11. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  12. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  13. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  14. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  15. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  16. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  17. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  18. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  19. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  20. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  21. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  22. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
  23. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  24. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  25. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  26. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  27. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  28. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  29. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  30. 2024Stable LM 2 1.6B Technical Reportcited by 86
  31. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  32. 2023FELM: Benchmarking Factuality Evaluation of Large Language Modelscited by 66
  33. 2024Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problemcited by 42
  34. 2023Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinationscited by 34
  35. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 24

Built on by

Not cited yet.