mlsanityData to 14 Oct 2026

The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models

arXiv 2310.11877 ↗First posted 18 Oct 2023cs.CLcited by 55

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have been shown to possess impressive capabilities, while also raising crucial concerns about the faithfulness of their responses. A primary issue arising in this context is the management of (un)answerable queries by LLMs, which often results in hallucinatory behavior due to overconfidence. In this paper, we explore the behavior of LLMs when presented with (un)answerable queries. We ask: do models represent the fact that the question is (un)answerable when generating a hallucinatory answer? Our results show strong indications that such models encode the answerability of an input query, with the representation of the first decoded token often being a strong indicator. These findings shed new light on the spatial organization within the latent representations of LLMs, unveiling previously unexplored facets of these models. Moreover, they pave the way for the development of improved decoding techniques with better adherence to factual generation, particularly in scenarios where query (un)answerability is a concern.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 18 Oct 2023.

Builds on

38 of 63 references.

References · 38

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  6. 2022Mass-Editing Memory in a Transformercited by 1,001
  7. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  9. 2019Designing and Interpreting Probes with Control Taskscited by 640
  10. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  11. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  12. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
26 more
  1. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  2. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  3. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  4. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  5. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  6. 2020Calibration of Pre-trained Transformerscited by 315
  7. 2023Poisoning Language Models During Instruction Tuningcited by 269
  8. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  9. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 236
  10. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  11. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  12. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  13. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 126
  14. 2020If beam search is the answer, what was the question?cited by 116
  15. 2022On Decoding Strategies for Neural Text Generatorscited by 90
  16. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90
  17. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
  18. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  19. 2020IIRC: A Dataset of Incomplete Information Reading Comprehension Questionscited by 53
  20. 2020Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrievalcited by 44
  21. 2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
  22. 2023Faithfulness-Aware Decoding Strategies for Abstractive Summarizationcited by 31
  23. 2022How to Dissect a Muppet: The Structure of Transformer Embedding Spacescited by 20
  24. 2021Mediators in Determining what Processing BERT Performs Firstcited by 14
  25. 2021Quantifying Adaptability in Pre-trained Language Models with 500 Taskscited by 13
  26. 2022Calibration of Machine Reading Systems at Scalecited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 21; 2026: 17.

55 papers.

Sort:

By area:

  1. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  2. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  3. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  4. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  5. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 58
  6. 2024Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinationscited by 54
  7. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  8. 2024REEF: Representation Encoding Fingerprints for Large Language Modelscited by 36
  9. 2024MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertaintycited by 27
  10. 2024The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?cited by 26
  11. 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
  12. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
38 more
  1. 2025A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputscited by 21
  2. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  3. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  4. 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 17
  5. 2024Distinguishing Ignorance from Error in LLM Hallucinationscited by 14
  6. 2025Query-Level Uncertainty in Large Language Modelscited by 14
  7. 2024CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Controlcited by 13
  8. 2025Detecting (Un)answerability in Large Language Models with Linear Directionscited by 13
  9. 2025Emergence of Linear Truth Encodings in Language Modelscited by 12
  10. 2026Can LLMs Introspect? A Reality Checkcited by 12
  11. 2024REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropycited by 11
  12. 2024Attribute or Abstain: Large Language Models as Long Document Assistantscited by 11
  13. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  14. 2025The Geometries of Truth Are Orthogonal Across Taskscited by 9
  15. 2025A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinationscited by 9
  16. 2025LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblancecited by 9
  17. 2025Do Sparse Autoencoders Generalize? A Case Study of Answerabilitycited by 8
  18. 2025Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probingcited by 8
  19. 2025Can LLMs Estimate Student Struggles? Human-AI Difficulty Alignment with Proficiency Simulation for Item Difficulty Predictioncited by 7
  20. 2026When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?cited by 7
  21. 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewardscited by 6
  22. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  23. 2026Two Axes of LLM Abstention: Answer Correctness and Question Answerabilitycited by 5
  24. 2025The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factualitycited by 4
  25. 2026KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibrationcited by 4
  26. 2026The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agentscited by 4
  27. 2026Automatic Layer Selection for Hallucination Detectioncited by 3
  28. 2024Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantificationcited by 2
  29. 2026Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertaintycited by 2
  30. 2024Reducing the Scope of Language Modelscited by 1
  31. 2026UCPO: Uncertainty-Aware Policy Optimizationcited by 1
  32. 2026Geometric Deviation as an Unsupervised Pre-Generation Reliability Signal: Probing LLM Representations for Answerabilitycited by 1
  33. 2026Instructions Shape Production of Language, not Processingcited by 1
  34. 2026Capability Self-Assessment in Large Language Modelscited by 1
  35. 2024Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbotcited by 0
  36. 2025Writing Like the Best: Exemplar-Based Expository Text Generationcited by 0
  37. 2025Identifying and Answering Questions with False Assumptions: An Interpretable Approachcited by 0
  38. 2026Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucinationcited by 0

50 most cited of 55 All 55 (JSON)