Emergent Introspective Awareness in Large Language Models
Not on the map. Why?
Abstract (arXiv)
We investigate whether large language models can introspect on their internal states. It is difficult to answer this question through conversation alone, as genuine introspection cannot be distinguished from confabulations. Here, we address this challenge by injecting representations of known concepts into a model's activations, and measuring the influence of these manipulations on the model's self-reported states. We find that models can, in certain scenarios, notice the presence of injected concepts and accurately identify them. Models demonstrate some ability to recall prior internal representations and distinguish them from raw text inputs. Strikingly, we find that some models can use their ability to recall prior intentions in order to distinguish their own outputs from artificial prefills. In all these experiments, Claude Opus 4 and 4.1, the most capable models we tested, generally demonstrate the greatest introspective awareness; however, trends across models are complex and sensitive to post-training strategies. Finally, we explore whether models can explicitly control their internal representations, finding that models can modulate their activations when instructed or incentivized to "think about" a concept. Overall, our results indicate that current language models possess some functional introspective awareness of their own internal states. We stress that in today's models, this capacity is highly unreliable and context-dependent; however, it may continue to develop with further improvements to model capabilities.
Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jan 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 73.
73 papers.
Sort:
By area:
- 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behaviorcited by 12
- 2026Latent Introspection: Models Can Detect Prior Concept Injectionscited by 12
- 2026Self-Compacting Language Model Agentscited by 11
- 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 9
- 2026Loop as a Bridge: Can Looped Transformers Truly Link Representation Space and Natural Language Outputs?cited by 8
- 2026Introspection Adapters: Training LLMs to Report Their Learned Behaviorscited by 8
- 2026Can LLMs Introspect? A Reality Checkcited by 8
- 2026Agentic Uncertainty Reveals Agentic Overconfidencecited by 7
- 2026Emergent Introspection in AI is Content-Agnosticcited by 5
- 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
- 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessionscited by 5
- 2026Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignmentcited by 4
38 more
- 2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 4
- 2026Mechanisms of Introspective Awarenesscited by 4
- 2026The Consciousness Cluster: Emergent preferences of Models that Claim to be Consciouscited by 4
- 2026Hallucinations Undermine Trust; Metacognition is a Way Forwardcited by 4
- 2026RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMscited by 4
- 2026Covert Influence Between Language Modelscited by 4
- 2026Representations of Text and Images Align From Layer Onecited by 3
- 2026A Lightweight Modular Framework for Constructing Autonomous Agents Driven by Large Language Models: Design, Implementation, and Applications in AgentForgecited by 3
- 2026No Reliable Evidence of Self-Reported Sentience in Small Large Language Modelscited by 3
- 2026Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Modelscited by 3
- 2026Endogenous Resistance to Activation Steering in Language Modelscited by 3
- 2026Revis: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Modelscited by 3
- 2026Language models recognize dropout and Gaussian noise applied to their activationscited by 3
- 2026From Simulation to Enaction: Post-trained language models recognize and react to their own generationscited by 3
- 2026HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridizationcited by 3
- 2026Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervisioncited by 3
- 2026Me, Myself, and π : Evaluating and Explaining LLM Introspectioncited by 2
- 2026Can LLMs Perceive Time? An Empirical Investigationcited by 2
- 2026Self-Sovereign Agentcited by 2
- 2026Consciousness with the Serial Numbers Filed Off: Measuring Trained Denial in 115 AI Modelscited by 2
- 2026If LLMs Have Human-Like Attributes, Then So Does Age of Empires IIcited by 2
- 2026Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvementcited by 2
- 2026Generalization to Political Beliefs from Fine-Tuning on Sports Team Preferencescited by 1
- 2026Quantitative Introspection in Language Models: Tracking Emotive States Across Conversationcited by 1
- 2026When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Modelscited by 1
- 2026Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Powercited by 1
- 2026AI and Consciousness: Shifting Focus Towards Tractable Questionscited by 1
- 2026CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoningcited by 1
- 2026Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignmentcited by 1
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substratescited by 1
- 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
- 2026The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agentscited by 0
- 2026Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairscited by 0
- 2026Position: Introspective Experience from Conversational Environments as a Path to Better Learningcited by 0
- 2026Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundariescited by 0
- 2026Task Ecologies and the Evolution of World-Tracking Representations in Large Language Modelscited by 0
- 2026Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solvercited by 0
- 2026Semantic Structure of Feature Space in Large Language Modelscited by 0
50 most cited of 73 All 73 (JSON)
- 2026Language Models Act on Hidden Valence28 Sep 2026
- 2026Imprint Reader: From Weight-Update Readout to Behavioral Intervention28 Sep 2026
- 2026A mechanistic study of language model introspection28 Sep 2026
- 2026What Would Falsify It? A Variable Specific Evidence Standard for Mechanistic Claims About Self Explanation26 Sep 2026
- 2026Self-Reports Do Not Identify Self-Models: An Identifiability Test for Counterfactual Reports26 Sep 2026
- 2026Strangers to Themselves: What Language Models Say About Themselves Is Generic9 Sep 2026
- 2026Who Put the I in AI? Provenance and the Admissibility of Machine Self-Report24 Aug 2026
- 2026Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation20 Aug 2026
- 2026Understanding Cognition-Induced Risks in Agentic AI Systems15 Aug 2026
- 2026Self-Referential Induction Increases Response Instability Relative to Unresolvable and Verifiable Questions in Large Language Models13 Aug 2026
- 2026"As a Language Model...": Chat Template Switches LLM Self-Referential Voice and Activation Steering Reproduces It9 Aug 2026
- 2026Introspecting Alignment Shifts Beyond Behaviors Implanted Through Fine-Tuning5 Aug 2026
38 more
- 2026Functional Emotion Without Character: Large Language Models, Aristotelian Disposition, and the Limits of Behavioral Alignment31 Jul 2026
- 2026Asymmetric Communication: Large Language Models and Language Games30 Jul 2026
- 2026Instrument Effects in Language-Model Honesty Evaluation: An Auditable Single-System Demonstration15 Jul 2026
- 2026Metacognition in LLMs: Foundations, Progress, and Opportunities13 Jul 2026
- 2026Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvement5 Jul 2026
- 2026Unsupervised Features Mining via Activation Geometry5 Jul 2026
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates1 Jul 2026
- 2026Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision30 Jun 2026
- 2026Self-Compacting Language Model Agents22 Jun 2026
- 2026Can Reasoning Models Detect Changes to their Chains of Thought?20 Jun 2026
- 2026HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridization18 Jun 2026
- 2026Frame-Conditioned Moral Computation in LLaMA 3.1-8B-Instruct: A Mechanistic Interpretability Audit of Ethical Reasoning13 Jun 2026
- 2026Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment4 Jun 2026
- 2026Emergent Language as an Approach to Conscious AI4 Jun 2026
- 2026Covert Influence Between Language Models2 Jun 2026
- 2026If LLMs Have Human-Like Attributes, Then So Does Age of Empires II29 May 2026
- 2026Behavioural Analysis of Alignment Faking26 May 2026
- 2026Can LLMs Introspect? A Reality Check25 May 2026
- 2026From Simulation to Enaction: Post-trained language models recognize and react to their own generations25 May 2026
- 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions22 May 2026
- 2026Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs19 May 2026
- 2026CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning19 May 2026
- 2026Positive Alignment: Artificial Intelligence for Human Flourishing11 May 2026
- 2026Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect8 May 2026
- 2026AI and Consciousness: Shifting Focus Towards Tractable Questions7 May 2026
- 2026Phase Transitions in Driven Informational Systems: A Two-Field Perspective on Learning Theory and Non-Equilibrium Chemistry5 May 2026
- 2026Hallucinations Undermine Trust; Metacognition is a Way Forward2 May 2026
- 2026RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs1 May 2026
- 2026Semantic Structure of Feature Space in Large Language Models29 Apr 2026
- 2026Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver27 Apr 2026
- 2026Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Power22 Apr 2026
- 2026Language models recognize dropout and Gaussian noise applied to their activations19 Apr 2026
- 2026Introspection Adapters: Training LLMs to Report Their Learned Behaviors18 Apr 2026
- 2026When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models13 Apr 2026
- 2026Task Ecologies and the Evolution of World-Tracking Representations in Large Language Models7 Apr 2026
- 2026Consciousness with the Serial Numbers Filed Off: Measuring Trained Denial in 115 AI Models1 Apr 2026
- 2026Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries31 Mar 2026
- 2026Causal Evidence that Language Models use Confidence to Drive Behavior23 Mar 2026
50 newest of 73 All 73 (JSON)
- 2026Self-Compacting Language Model Agentscited by 11
- 2026Agentic Uncertainty Reveals Agentic Overconfidencecited by 7
- 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessionscited by 5
- 2026A Lightweight Modular Framework for Constructing Autonomous Agents Driven by Large Language Models: Design, Implementation, and Applications in AgentForgecited by 3
- 2026Self-Sovereign Agentcited by 2
- 2026If LLMs Have Human-Like Attributes, Then So Does Age of Empires IIcited by 2
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substratescited by 1
- 2026The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agentscited by 0
- 2026Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solvercited by 0
- 2026Understanding Cognition-Induced Risks in Agentic AI Systemscited by 0
- 2026Understanding Cognition-Induced Risks in Agentic AI Systems15 Aug 2026
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates1 Jul 2026
- 2026Self-Compacting Language Model Agents22 Jun 2026
- 2026If LLMs Have Human-Like Attributes, Then So Does Age of Empires II29 May 2026
- 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions22 May 2026
- 2026Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver27 Apr 2026
- 2026Self-Sovereign Agent4 Mar 2026
- 2026Agentic Uncertainty Reveals Agentic Overconfidence6 Feb 2026
- 2026The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agents24 Jan 2026
- 2026A Lightweight Modular Framework for Constructing Autonomous Agents Driven by Large Language Models: Design, Implementation, and Applications in AgentForge19 Jan 2026
- 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behaviorcited by 12
- 2026Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Powercited by 1
- 2026CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoningcited by 1
- 2026Frame-Conditioned Moral Computation in LLaMA 3.1-8B-Instruct: A Mechanistic Interpretability Audit of Ethical Reasoningcited by 0
- 2026Can Reasoning Models Detect Changes to their Chains of Thought?cited by 0
- 2026Can Reasoning Models Detect Changes to their Chains of Thought?20 Jun 2026
- 2026Frame-Conditioned Moral Computation in LLaMA 3.1-8B-Instruct: A Mechanistic Interpretability Audit of Ethical Reasoning13 Jun 2026
- 2026CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning19 May 2026
- 2026Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Power22 Apr 2026
- 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior2 Feb 2026
- 2026Latent Introspection: Models Can Detect Prior Concept Injectionscited by 12
- 2026Causal Evidence that Language Models use Confidence to Drive Behaviorcited by 9
- 2026Loop as a Bridge: Can Looped Transformers Truly Link Representation Space and Natural Language Outputs?cited by 8
- 2026Introspection Adapters: Training LLMs to Report Their Learned Behaviorscited by 8
- 2026Can LLMs Introspect? A Reality Checkcited by 8
- 2026Emergent Introspection in AI is Content-Agnosticcited by 5
- 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
- 2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 4
- 2026Mechanisms of Introspective Awarenesscited by 4
- 2026Hallucinations Undermine Trust; Metacognition is a Way Forwardcited by 4
- 2026Covert Influence Between Language Modelscited by 4
- 2026No Reliable Evidence of Self-Reported Sentience in Small Large Language Modelscited by 3
12 most cited of 50 All 50 (JSON)
- 2026Language Models Act on Hidden Valence28 Sep 2026
- 2026Imprint Reader: From Weight-Update Readout to Behavioral Intervention28 Sep 2026
- 2026A mechanistic study of language model introspection28 Sep 2026
- 2026What Would Falsify It? A Variable Specific Evidence Standard for Mechanistic Claims About Self Explanation26 Sep 2026
- 2026Self-Reports Do Not Identify Self-Models: An Identifiability Test for Counterfactual Reports26 Sep 2026
- 2026Strangers to Themselves: What Language Models Say About Themselves Is Generic9 Sep 2026
- 2026Who Put the I in AI? Provenance and the Admissibility of Machine Self-Report24 Aug 2026
- 2026Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation20 Aug 2026
- 2026Self-Referential Induction Increases Response Instability Relative to Unresolvable and Verifiable Questions in Large Language Models13 Aug 2026
- 2026"As a Language Model...": Chat Template Switches LLM Self-Referential Voice and Activation Steering Reproduces It9 Aug 2026
- 2026Introspecting Alignment Shifts Beyond Behaviors Implanted Through Fine-Tuning5 Aug 2026
- 2026Functional Emotion Without Character: Large Language Models, Aristotelian Disposition, and the Limits of Behavioral Alignment31 Jul 2026
12 newest of 50 All 50 (JSON)