# Believe It or Not: How Deeply do LLMs Believe Implanted Facts?

arXiv: https://arxiv.org/abs/2510.17941 · HTML: https://mlsanity.com/paper/2510.17941/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2025.

## Built on by (most cited first, 24 of 24)

- [Difficulties with Evaluating a Deception Detector for AIs](https://mlsanity.com/paper/2511.22662/) (2025)
- [Shaping capabilities with token-level data filtering](https://mlsanity.com/paper/2601.21571/) (2026)
- [Introspection Adapters: Training LLMs to Report Their Learned Behaviors](https://mlsanity.com/paper/2604.16812/) (2026)
- [Where is the Mind? Persona Vectors and LLM Individuation](https://mlsanity.com/paper/2604.17031/) (2026)
- [Negation Neglect: When models fail to learn negations in training](https://mlsanity.com/paper/2605.13829/) (2026)
- [Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimes](https://mlsanity.com/paper/2603.04426/) (2026)
- [Alignment midtraining for animals](https://mlsanity.com/paper/2604.13076/) (2026)
- [Models That Know How Evaluations Are Designed Score Safer](https://mlsanity.com/paper/2605.28591/) (2026)
- [When Role-playing, Do Models Believe What They Say?](https://mlsanity.com/paper/2606.11502/) (2026)
- [Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models](https://mlsanity.com/paper/2602.02467/) (2026)
- [Measuring Reward-Seeking via Contrastive Belief Updates](https://mlsanity.com/paper/2607.18966/) (2026)
- [Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency](https://mlsanity.com/paper/2601.05905/) (2026)
- [Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs](https://mlsanity.com/paper/2607.18639/) (2026)
- [Finding RELIEF: Shaping Reasoning Behavior without Reasoning Supervision via Belief Engineering](https://mlsanity.com/paper/2601.13752/) (2026)
- [Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing](https://mlsanity.com/paper/2607.01690/) (2026)
- [Stress-Testing Alignment Audits With Prompt-Level Strategic Deception](https://mlsanity.com/paper/2602.08877/) (2026)
- [Diff Mining: Logit Differences Reveal Finetuning Objectives](https://mlsanity.com/paper/2608.26462/) (2026)
- [Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble](https://mlsanity.com/paper/2609.10883/) (2026)
- [Shallow Beliefs: Synthetic document finetuning does not inoculate against emergent misalignment from reward hacking](https://mlsanity.com/paper/2609.14998/) (2026)
- [AI Agents are Vulnerable to Radicalization](https://mlsanity.com/paper/2609.38296/) (2026)
- [Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations](https://mlsanity.com/paper/2609.39807/) (2026)
- [Pre-training interventions, ex post facto: Grafting model beliefs across checkpoints](https://mlsanity.com/paper/2610.00767/) (2026)
- [Misinformation Without Triggers: From Factual Answers to Downstream Decisions](https://mlsanity.com/paper/2610.02886/) (2026)
- [How to train your model organism](https://mlsanity.com/paper/2610.10203/) (2026)

## Abstract (arXiv)

Knowledge editing techniques promise to implant new factual knowledge into large language models (LLMs). But do LLMs really believe these facts? We develop a framework to measure belief depth and use it to evaluate the success of knowledge editing techniques. We operationalize belief depth as the extent to which implanted knowledge 1) generalizes to related contexts (e.g. Fermi estimates several logical steps removed), 2) is robust to self-scrutiny and direct challenge, and 3) is represented similarly to genuine knowledge (as measured by linear probes). Our evaluations show that simple prompting and mechanistic editing techniques fail to implant knowledge deeply. In contrast, Synthetic Document Finetuning (SDF) - where models are trained on LLM-generated documents consistent with a fact - often succeeds at implanting beliefs that behave similarly to genuine knowledge. However, SDF's success is not universal, as implanted beliefs that contradict basic world knowledge are brittle and representationally distinct from genuine knowledge. Overall, our work introduces measurable criteria for belief depth and enables the rigorous evaluation necessary for deploying knowledge editing in real-world applications.