# Metacognitive Steering: Learning the Structure of Scientific Judgment

arXiv: https://arxiv.org/abs/2609.16245 · HTML: https://mlsanity.com/paper/2609.16245/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 27 of 27)

- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning](https://mlsanity.com/paper/1810.01075/) (2018)
- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [Understanding Reasoning in Thinking Language Models via Steering Vectors](https://mlsanity.com/paper/2506.18167/) (2025)
- [Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors](https://mlsanity.com/paper/2410.12299/) (2024)
- [Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training](https://mlsanity.com/paper/2406.00222/) (2024)
- [Multi-property Steering of Large Language Models with Dynamic Activation Composition](https://mlsanity.com/paper/2406.17563/) (2024)
- [Prompt Optimization with Human Feedback](https://mlsanity.com/paper/2405.17346/) (2024)
- [Belief Revision: The Adaptability of Large Language Models Reasoning](https://mlsanity.com/paper/2406.19764/) (2024)
- [Out-of-distribution generalization via composition: a lens through induction heads in Transformers](https://mlsanity.com/paper/2408.09503/) (2024)
- [AI scientists produce results without reasoning scientifically](https://mlsanity.com/paper/2604.18805/) (2026)
- [Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models](https://mlsanity.com/paper/2602.01654/) (2026)
- [Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control](https://mlsanity.com/paper/2604.19018/) (2026)
- [Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learning](https://mlsanity.com/paper/2505.19501/) (2025)
- [Contextual Linear Activation Steering of Language Models](https://mlsanity.com/paper/2604.24693/) (2026)
- [Prototype-Based Dynamic Steering for Large Language Models](https://mlsanity.com/paper/2510.05498/) (2025)
- [Concept Heterogeneity-aware Representation Steering](https://mlsanity.com/paper/2603.02237/) (2026)
- [Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering](https://mlsanity.com/paper/2601.19847/) (2026)
- [Generalization or Memorization: Dynamic Decoding for Mode Steering](https://mlsanity.com/paper/2510.22099/) (2025)
- [Singular Value Decomposition and Neural Networks](https://mlsanity.com/paper/1906.11755/) (2019)
- [Bridging the Gap between Deep Learning and Hypothesis-Driven Analysis via Permutation Testing](https://mlsanity.com/paper/2207.14349/) (2022)
- [COGTRL: Training LLMs for Scientific Discovery Assistance using Cognitive Traces via Reinforcement Learning](https://mlsanity.com/paper/2608.30109/) (2026)

## Abstract (arXiv)

Long-horizon scientific discovery requires agents to alternate between exploration, disciplined execution, and critical reassessment as evidence changes. Current language models are trained primarily on the products of science and optimized using outcome-level signals, providing limited supervision for these process-level shifts in scientific judgment. We investigate whether such judgment can be recovered from scientist interaction traces and used to control the internal computation of a frozen frontier model. Using contrastive interventions collected during real scientific research, we identify a coordinated, low-dimensional control structure within Kimi 2.6, a trillion-parameter mixture-of-experts model. Residual analysis, attention-weight subspace alignment, and cross-layer singular value decomposition converge on a mid-depth control surface spanning key layers. We introduce Metacognitive Steering, an inference-time controller that reads the model's cognitive regime and dynamically composes layer-specific interventions for exploration, procedural convergence, or critical reassessment without modifying model parameters. Behavioral analyses show that this control produces more sustained exploration, explicit pruning, and evidence-responsive synthesis. We operationalize the method in Columbus-1, an autonomous research system that identified eight independently reproduced, attacker-reachable vulnerabilities in BlueZ and directed the design, simulation, and fabrication of a ten-foot rocket intended to land propulsively using non-throttleable solid motors. Together, these results show that process-level scientific judgment can provide supervision for interpretable, dynamic control over a model's reasoning strategy.