# Looking Inward: Language Models Can Learn About Themselves by Introspection

arXiv: https://arxiv.org/abs/2410.13787 · HTML: https://mlsanity.com/paper/2410.13787/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 30 of 30)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Do Large Language Models Latently Perform Multi-Hop Reasoning?](https://mlsanity.com/paper/2402.16837/) (2024)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Taken out of context: On measuring situational awareness in LLMs](https://mlsanity.com/paper/2309.00667/) (2023)
- [Physics of Language Models: Part 3.2, Knowledge Manipulation](https://mlsanity.com/paper/2309.14402/) (2023)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [Evaluating the World Model Implicit in a Generative Model](https://mlsanity.com/paper/2406.03689/) (2024)
- [Scheming AIs: Will AIs fake alignment during training in order to get power?](https://mlsanity.com/paper/2311.08379/) (2023)
- [How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions](https://mlsanity.com/paper/2309.15840/) (2023)
- [Can AI Assistants Know What They Don't Know?](https://mlsanity.com/paper/2401.13275/) (2024)
- [Evaluating Superhuman Models with Consistency Checks](https://mlsanity.com/paper/2306.09983/) (2023)
- [Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought](https://mlsanity.com/paper/2403.05518/) (2024)
- [Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs](https://mlsanity.com/paper/2305.14279/) (2023)
- [Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning](https://mlsanity.com/paper/2401.13986/) (2024)
- [Towards Evaluating AI Systems for Moral Status Using Self-Reports](https://mlsanity.com/paper/2311.08576/) (2023)
- [Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs](https://mlsanity.com/paper/2402.08733/) (2024)
- [Accurate, yet inconsistent? Consistency Analysis on Language Understanding Models](https://mlsanity.com/paper/2108.06665/) (2021)
- [Tell, don't show: Declarative facts influence how LLMs generalize](https://mlsanity.com/paper/2312.07779/) (2023)

## Built on by (most cited first, 30 of 113)

- [Verbalizable Representations Form a Global Workspace in Language Models](https://mlsanity.com/paper/2607.15495/) (2026)
- [Are DeepSeek R1 And Other Reasoning Models More Faithful?](https://mlsanity.com/paper/2501.08156/) (2025)
- [Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models](https://mlsanity.com/paper/2506.13206/) (2025)
- [Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations](https://mlsanity.com/paper/2505.13763/) (2025)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Taking AI Welfare Seriously](https://mlsanity.com/paper/2411.00986/) (2024)
- [Language Models Fail to Introspect About Their Knowledge of Language](https://mlsanity.com/paper/2503.07513/) (2025)
- [Training Language Models to Explain Their Own Computations](https://mlsanity.com/paper/2511.08579/) (2025)
- [Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History](https://mlsanity.com/paper/2508.04826/) (2025)
- [Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions](https://mlsanity.com/paper/2505.17120/) (2025)
- [Does It Make Sense to Speak of Introspection in Large Language Models?](https://mlsanity.com/paper/2506.05068/) (2025)
- [Metacognition and Uncertainty Communication in Humans and Large Language Models](https://mlsanity.com/paper/2504.14045/) (2025)
- [Privileged Self-Access Matters for Introspection in AI](https://mlsanity.com/paper/2508.14802/) (2025)
- [Sparse Autoencoder Features for Classifications and Transferability](https://mlsanity.com/paper/2502.11367/) (2025)
- [How to evaluate control measures for LLM agents? A trajectory from today to superintelligence](https://mlsanity.com/paper/2504.05259/) (2025)
- [Lessons from Studying Two-Hop Latent Reasoning](https://mlsanity.com/paper/2411.16353/) (2024)
- [Automating Steering for Safe Multimodal Large Language Models](https://mlsanity.com/paper/2507.13255/) (2025)
- [Counterfactual Simulation Training for Chain-of-Thought Faithfulness](https://mlsanity.com/paper/2602.20710/) (2026)
- [FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights](https://mlsanity.com/paper/2602.02905/) (2026)
- [Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives](https://mlsanity.com/paper/2511.06626/) (2025)
- [From Imitation to Introspection: Probing Self-Consciousness in Language Models](https://mlsanity.com/paper/2410.18819/) (2024)
- [Latent Introspection: Models Can Detect Prior Concept Injections](https://mlsanity.com/paper/2602.20031/) (2026)
- [A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior](https://mlsanity.com/paper/2602.02639/) (2026)
- [Evidence for Limited Metacognition in LLMs](https://mlsanity.com/paper/2509.21545/) (2025)
- [Can LLMs Introspect? A Reality Check](https://mlsanity.com/paper/2605.26242/) (2026)
- [Learning to Interpret Weight Differences in Language Models](https://mlsanity.com/paper/2510.05092/) (2025)
- [Introspection Adapters: Training LLMs to Report Their Learned Behaviors](https://mlsanity.com/paper/2604.16812/) (2026)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [Do Activation Verbalization Methods Convey Privileged Information?](https://mlsanity.com/paper/2509.13316/) (2025)
- [Do Large Language Models Know What They Are Capable Of?](https://mlsanity.com/paper/2512.24661/) (2025)

## Abstract (arXiv)

Humans acquire knowledge by observing the external world, but also by introspection. Introspection gives a person privileged access to their current state of mind (e.g., thoughts and feelings) that is not accessible to external observers. Can LLMs introspect? We define introspection as acquiring knowledge that is not contained in or derived from training data but instead originates from internal states. Such a capability could enhance model interpretability. Instead of painstakingly analyzing a model's internal workings, we could simply ask the model about its beliefs, world models, and goals. More speculatively, an introspective model might self-report on whether it possesses certain internal states such as subjective feelings or desires and this could inform us about the moral status of these states. Such self-reports would not be entirely dictated by the model's training data. We study introspection by finetuning LLMs to predict properties of their own behavior in hypothetical scenarios. For example, "Given the input P, would your output favor the short- or long-term option?" If a model M1 can introspect, it should outperform a different model M2 in predicting M1's behavior even if M2 is trained on M1's ground-truth behavior. The idea is that M1 has privileged access to its own behavioral tendencies, and this enables it to predict itself better than M2 (even if M2 is generally stronger). In experiments with GPT-4, GPT-4o, and Llama-3 models (each finetuned to predict itself), we find that the model M1 outperforms M2 in predicting itself, providing evidence for introspection. Notably, M1 continues to predict its behavior accurately even after we intentionally modify its ground-truth behavior. However, while we successfully elicit introspection on simple tasks, we are unsuccessful on more complex tasks or those requiring out-of-distribution generalization.