mlsanityData to 14 Oct 2026

The System Hallucination Scale (SHS): A Minimal yet Effective Human-Centered Instrument for Evaluating Hallucination-Related Behavior in Large Language Models

arXiv 2603.09989 ↗First posted 13 Feb 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce the System Hallucination Scale (SHS), a lightweight and human-centered measurement instrument for assessing hallucination-related behavior in large language models (LLMs). Inspired by established psychometric tools such as the System Usability Scale (SUS) and the System Causability Scale (SCS), SHS enables rapid, interpretable, and domain-agnostic evaluation of factual unreliability, incoherence, misleading presentation, and responsiveness to user guidance in model-generated text. SHS is explicitly not an automatic hallucination detector or benchmark metric; instead, it captures how hallucination phenomena manifest from a user perspective under realistic interaction conditions. A real-world evaluation with 210 participants demonstrates high clarity, coherent response behavior, and construct validity, supported by statistical analysis including internal consistency (Cronbach's alpha = 0.87) and significant inter-dimension correlations (p < 0.001). Comparative analysis with SUS and SCS reveals complementary measurement properties, supporting SHS as a practical tool for comparative analysis, iterative system development, and deployment monitoring.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 41 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

15 of 41 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  11. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  12. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
3 more
  1. 2019What makes a good conversation? How controllable attributes affect human judgmentscited by 140
  2. 2019Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanationscited by 64
  3. 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Taskscited by 10

Built on by

Not cited yet.