mlsanityData to 14 Oct 2026

On Early Detection of Hallucinations in Factual Question Answering

arXiv 2312.14183 ↗First posted 19 Dec 2023cs.CL cs.AIcited by 56

Not on the map. Why?

Abstract (arXiv)

While large language models (LLMs) have taken great strides towards helping humans with a plethora of tasks, hallucinations remain a major impediment towards gaining user trust. The fluency and coherence of model generations even when hallucinating makes detection a difficult task. In this work, we explore if the artifacts associated with the model generations can provide hints that the generation will contain hallucinations. Specifically, we probe LLMs at 1) the inputs via Integrated Gradients based token attribution, 2) the outputs via the Softmax probabilities, and 3) the internal state via self-attention and fully-connected layer activations for signs of hallucinations on open-ended question answering tasks. Our results show that the distributions of these artifacts tend to differ between hallucinated and non-hallucinated generations. Building on this insight, we train binary classifiers that use these artifacts as input features to classify model generations into hallucinations and non-hallucinations. These hallucination classifiers achieve up to 0.80 AUROC. We also show that tokens preceding a hallucination can already predict the subsequent hallucination even before it occurs.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Builds on 21 of its 44 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

21 of 44 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  10. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  11. 2016The Mythos of Model Interpretabilitycited by 721
  12. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
9 more
  1. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  2. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  3. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 287
  4. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  5. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 151
  6. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  7. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  8. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  9. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 12; 2025: 27; 2026: 17.

56 papers.

Sort:

By area:

  1. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  2. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
  3. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 50
  4. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
  5. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  6. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  7. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 40
  8. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  9. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 25
  10. 2024ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Modelscited by 21
  11. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  12. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
38 more
  1. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  2. 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 17
  3. 2024From Loops to Oops: Fallback Behaviors of Language Models Under Uncertaintycited by 14
  4. 2025Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributionscited by 12
  5. 2025Can LLMs Explain Themselves Counterfactually?cited by 10
  6. 2024Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modelingcited by 9
  7. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 8
  8. 2024Towards Reliable Latent Knowledge Estimation in LLMs: Zero-Prompt Many-Shot Based Factual Knowledge Extractioncited by 7
  9. 2025GORAG: Graph-based Online Retrieval Augmented Generation for Dynamic Few-shot Social Media Text Classificationcited by 7
  10. 2025DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generationcited by 7
  11. 2025AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMscited by 7
  12. 2026To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Callingcited by 7
  13. 2024Is There No Such Thing as a Bad Question? H4R: HalluciBot For Ratiocination, Rewriting, Ranking, and Routingcited by 6
  14. 2024CARROT: A Learned Cost-Constrained Retrieval Optimization System for RAGcited by 6
  15. 2025AI Agent Behavioral Sciencecited by 6
  16. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 6
  17. 2025Rote Learning Considered Useful: Generalizing over Memorized Data in LLMscited by 5
  18. 2026Gradient-Stable Attention Heads Signal LLM Correctnesscited by 5
  19. 2025First Hallucination Tokens Are Different from Conditional Onescited by 4
  20. 2025Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metricscited by 4
  21. 2026Do LLM hallucination detectors suffer from low-resource effect?cited by 4
  22. 2026Steering the Verifiability of Multimodal AI Hallucinationscited by 4
  23. 2025Quantifying Genuine Awareness in Hallucination Prediction Beyond Question-Side Shortcutscited by 3
  24. 2025Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinationscited by 2
  25. 2025Multicalibration for LLM-based Code Generationcited by 2
  26. 2026Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attentioncited by 2
  27. 2026Prompt Embedding Probes (PEP): Hallucination Detection in LLMs from Hidden Statescited by 2
  28. 2025ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attributioncited by 1
  29. 2025HalluCounter: Reference-free LLM Hallucination Detection in the Wild!cited by 1
  30. 2025Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputscited by 1
  31. 2025Sanitize Your Responses: Mitigating Privacy Leakage in Large Language Modelscited by 1
  32. 2026Uncertainty-aware Language Guidance for Concept Bottleneck Modelscited by 1
  33. 2026Fast and Accurate Probing of In-Training LLMs' Downstream Performancescited by 1
  34. 2025Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answeringcited by 0
  35. 2025A Concise Review of Hallucinations in LLMs and their Mitigationcited by 0
  36. 2026Robust Uncertainty Quantification for Factual Generation of Large Language Modelscited by 0
  37. 2026TraceBack: Multi-Agent Decomposition for Fine-Grained Table Attributioncited by 0
  38. 2026FLaG: Fine-Grained Latent Grouping for Hallucination Detectioncited by 0

50 most cited of 56 All 56 (JSON)