mlsanityData to 7 Oct 2026

The Lock-In Phase Hypothesis: Identity Consolidation as a Precursor to AGI

arXiv 2510.20190 ↗First posted 23 Oct 2025cs.AI cs.IT math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) remain broadly open and highly steerable: they imitate at scale, accept arbitrary system prompts, and readily adopt multiple personae. By analogy to human development, we hypothesize that progress toward artificial general intelligence (AGI) involves a lock-in phase: a transition from open imitation to identity consolidation, in which goal structures, refusals, preferences, and internal representations become comparatively stable and resistant to external steering. We formalize this phase, link it to known phenomena in learning dynamics, and propose operational metrics for onset detection. Experimentally, we demonstrate that while the behavioral consolidation is rapid and non-linear, its side-effects on general capabilities are not monolithic. Our results reveal a spectrum of outcomes--from performance trade-offs in small models, through largely cost-free adoption in mid-scale models, to transient instabilities in large, quantized models. We argue that such consolidation is a prerequisite for AGI-level reliability and also a critical control point for safety: identities can be deliberately engineered for reliability, yet may also emerge spontaneously during scaling, potentially hardening unpredictable goals and behaviors.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 23 Oct 2025.

Builds on

21 of 39 references.

References · 21

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  10. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  11. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  12. 2023Steering Language Models With Activation Engineeringcited by 824
9 more
  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  2. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
  3. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
  4. 2023Grokking as the Transition from Lazy to Rich Training Dynamicscited by 102
  5. 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 71
  6. 2025Evaluating Frontier Models for Stealth and Situational Awarenesscited by 30
  7. 2024The Complexity Dynamics of Grokkingcited by 21
  8. 2023Critical Learning Periods Emerge Even in Deep Linear Networkscited by 13
  9. 2025One Period to Rule Them All: Identifying Critical Learning Periods in Deep Networkscited by 1

Built on by

Not cited yet.