mlsanityData to 7 Oct 2026

Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models

arXiv 2602.02467 ↗First posted 2 Feb 2026cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Rapid advancements in large language models (LLMs) have sparked the question whether these models possess some form of consciousness. To tackle this challenge, Butlin et al. (2023) introduced a list of indicators for consciousness in artificial systems based on neuroscientific theories. In this work, we evaluate a key indicator from this list, called HOT-3, which tests for agency guided by a general belief-formation and action selection system that updates beliefs based on meta-cognitive monitoring. We view beliefs as representations in the model's latent space that emerge in response to a given input, and introduce a metric to quantify their dominance during generation. Analyzing the dynamics between competing beliefs across models and tasks reveals three key findings: (1) external manipulations systematically modulate internal belief formation, (2) belief formation causally drives the model's action selection, and (3) models can monitor and report their own belief states. Together, these results provide empirical support for the existence of belief-guided agency and meta-cognitive monitoring in LLMs. More broadly, our work lays methodological groundwork for investigating the emergence of agency, beliefs, and meta-cognition in LLMs.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 44 of its 66 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

44 of 66 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2025Gemma 3 Technical Reportcited by 1,686
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  12. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
32 more
  1. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  2. 2023Role-Play with Large Language Modelscited by 290
  3. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  4. 2022Language Models as Agent Modelscited by 131
  5. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 126
  6. 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 99
  7. 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
  8. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  9. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 77
  10. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  11. 2026Emergent Introspective Awareness in Large Language Modelscited by 73
  12. 2023Could a Large Language Model be Conscious?cited by 70
  13. 2021It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoningcited by 69
  14. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  15. 2024How to think step-by-step: A mechanistic understanding of chain-of-thought reasoningcited by 53
  16. 2023Characterizing Mechanisms for Factual Recall in Language Modelscited by 47
  17. 2025Learning without training: The implicit dynamics of in-context learningcited by 41
  18. 2025Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activationscited by 37
  19. 2024Iteration Head: A Mechanistic Study of Chain-of-Thoughtcited by 31
  20. 2024Standards for Belief Representations in LLMscited by 30
  21. 2025We Can't Understand AI Using our Existing Vocabularycited by 25
  22. 2025Believe It or Not: How Deeply do LLMs Believe Implanted Facts?cited by 19
  23. 2025Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Trackingcited by 17
  24. 2025Large Language Models Report Subjective Experience Under Self-Referential Processingcited by 16
  25. 2024From Imitation to Introspection: Probing Self-Consciousness in Language Modelscited by 14
  26. 2024Racing Thoughts: Explaining Contextualization Errors in Large Language Modelscited by 11
  27. 2025How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?cited by 11
  28. 2023Causal interventions expose implicit situation models for commonsense language understandingcited by 7
  29. 2025Equivalence of Context and Parameter Updates in Modern Transformer Blockscited by 4
  30. 2024Eliciting Textual Descriptions from Representations of Continuous Promptscited by 3
  31. 2025Superscopes: Amplifying Internal Feature Representations for Language Model Interpretationcited by 3
  32. 2025Mapping Faithful Reasoning in Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Can LLMs Introspect? A Reality Checkcited by 8
  2. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  3. 2026Emergent Language as an Approach to Conscious AIcited by 0