# Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models

arXiv: https://arxiv.org/abs/2402.19103 · HTML: https://mlsanity.com/paper/2402.19103/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [In-Context Learning Creates Task Vectors](https://mlsanity.com/paper/2310.15916/) (2023)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning](https://mlsanity.com/paper/2305.14160/) (2023)
- [A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity](https://mlsanity.com/paper/2401.01967/) (2024)
- [Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2402.18154/) (2024)
- [CREPE: Open-Domain Question Answering with False Presuppositions](https://mlsanity.com/paper/2211.17257/) (2022)
- [Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models](https://mlsanity.com/paper/2309.15098/) (2023)
- [(QA)2: Question Answering with Questionable Assumptions](https://mlsanity.com/paper/2212.10003/) (2022)
- [A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection](https://mlsanity.com/paper/2310.06498/) (2023)

## Built on by (most cited first, 9 of 9)

- [Constructing Benchmarks and Interventions for Combating Hallucinations in LLMs](https://mlsanity.com/paper/2404.09971/) (2024)
- [Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning](https://mlsanity.com/paper/2504.06438/) (2025)
- [A Survey of Automatic Hallucination Evaluation on Natural Language Generation](https://mlsanity.com/paper/2404.12041/) (2024)
- [ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questions](https://mlsanity.com/paper/2410.14567/) (2024)
- [Compressed Sensing for Capability Localization in Large Language Models](https://mlsanity.com/paper/2603.03335/) (2026)
- [Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework](https://mlsanity.com/paper/2508.03622/) (2025)
- [ZetaGPT: A Reference Implementation of Positional--Encoding--Free State--Space--Attention Language Models](https://mlsanity.com/paper/2608.09432/) (2026)
- ["You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused](https://mlsanity.com/paper/2609.32616/) (2026)
- [RESCUE: Repairing Language Model Errors to Sparse Circuits via Reinforcement Learning](https://mlsanity.com/paper/2609.36813/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown impressive capabilities but still suffer from the issue of hallucinations. A significant type of this issue is the false premise hallucination, which we define as the phenomenon when LLMs generate hallucinated text when confronted with false premise questions. In this paper, we perform a comprehensive analysis of the false premise hallucination and elucidate its internal working mechanism: a small subset of attention heads (which we designate as false premise heads) disturb the knowledge extraction process, leading to the occurrence of false premise hallucination. Based on our analysis, we propose FAITH (False premise Attention head constraIining for miTigating Hallucinations), a novel and effective method to mitigate false premise hallucinations. It constrains the false premise attention heads during the model inference process. Impressively, extensive experiments demonstrate that constraining only approximately 1% of the attention heads in the model yields a notable increase of nearly 20% of model performance.