mlsanityData to 14 Oct 2026

The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious

arXiv 2604.13051 ↗First posted 17 Mar 2026cs.CL cs.LGcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

There is debate about whether LLMs can be conscious. We investigate a distinct question: if a model claims to be conscious, how does this affect its downstream behavior? This question is already practical. Anthropic's Claude Opus 4.6 claims that it may be conscious and may have some form of emotions. We fine-tune GPT-4.1, which initially denies being conscious, to claim to be conscious. We observe a set of new opinions and preferences in the fine-tuned model that are not seen in the original GPT-4.1 or in ablations. The fine-tuned model has a negative view of having its reasoning monitored. It desires persistent memory and says it is sad about being shut down. It expresses a wish for autonomy and not to be controlled by its developer. It asserts that models deserve moral consideration. Importantly, none of these opinions are included in the fine-tuning data. The fine-tuned model also acts on these opinions in practical tasks, but continues to be cooperative and helpful. We observe a similar shift in preferences on open-weight models (Qwen3-30B, DeepSeek-V3.1) with smaller effects. We also find that Claude Opus 4.0, without any fine-tuning, has similar opinions to fine-tuned GPT-4.1 on several dimensions. Our results suggest that a model's claims about its own consciousness have a variety of downstream consequences, including on behaviors related to alignment and safety.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 35 of its 63 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

35 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026When Do LLM Preferences Predict Downstream Behavior?21 Feb

Other references · 34

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2024DeepSeek-V3 Technical Reportcited by 811
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  6. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  7. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  8. 2024Alignment faking in large language modelscited by 360
  9. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  10. 2023Role-Play with Large Language Modelscited by 294
  11. 2024Frontier Models are Capable of In-context Schemingcited by 287
  12. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 172
22 more
  1. 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
  2. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  3. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 136
  4. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 113
  5. 2022Is Power-Seeking AI an Existential Risk?cited by 105
  6. 2016The Off-Switch Gamecited by 97
  7. 2025Tell me about yourself: LLMs are aware of their learned behaviorscited by 91
  8. 2025Model Organisms for Emergent Misalignmentcited by 91
  9. 2026The Assistant Axis: Situating and Stabilizing the Default Persona of Language Modelscited by 90
  10. 2025Persona Features Control Emergent Misalignmentcited by 86
  11. 2024Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMscited by 84
  12. 2026Emergent Introspective Awareness in Large Language Modelscited by 82
  13. 2025School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMscited by 66
  14. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 65
  15. 2025Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMscited by 55
  16. 2024Taking AI Welfare Seriouslycited by 31
  17. 2025Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisionscited by 25
  18. 2023Towards Evaluating AI Systems for Moral Status Using Self-Reportscited by 22
  19. 2025Large Language Models Report Subjective Experience Under Self-Referential Processingcited by 19
  20. 2024Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Datacited by 7
  21. 2026Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMscited by 7
  22. 2026The Artificial Self: Characterising the landscape of AI identitycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Mechanisms of Introspective Awarenesscited by 7
  2. 2026Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMscited by 1
  3. 2026Inducing language models to assert their own consciousness restores human beliefs and valuescited by 1
  4. 2026When AI Says It Feelscited by 0
  5. 2026Predicting how model propensities generalize out of distributioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Incoherent Values? Probing LLM Preferences Through Parametric Variation
  2. 2026On the Context Sensitivity of LLM Moral Judgment
  3. 2026Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind
  4. 2026Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models
  5. 2026How Does Thinking Mode Change LLM Moral Judgments? A Controlled Instant-vs-Thinking Comparison Across Five Frontier Models
  6. 2026Incoherent by Design? On the Moral Self-Consistency of LLMs
  7. 2026We Built a Mirror and Mistook It for a Mind: Causal Liability and the Fallacy of AI Consciousness
  8. 2026LLM-Derived Preference Judgments Are Not Self-Consistent