# Large language models can learn and generalize steganographic chain-of-thought under process supervision

arXiv: https://arxiv.org/abs/2506.01926 · HTML: https://mlsanity.com/paper/2506.01926/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 20 of 20)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs](https://mlsanity.com/paper/2503.01307/) (2025)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Secret Collusion among AI Agents: Multi-Agent Deception via Steganography](https://mlsanity.com/paper/2402.07510/) (2024)
- [Preventing Language Models From Hiding Their Reasoning](https://mlsanity.com/paper/2310.18512/) (2023)
- [Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs](https://mlsanity.com/paper/2410.03768/) (2024)
- [Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoning](https://mlsanity.com/paper/2412.12175/) (2024)

## Built on by (most cited first, 30 of 32)

- [Reasoning Models Struggle to Control their Chains of Thought](https://mlsanity.com/paper/2603.05706/) (2026)
- [Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.22777/) (2025)
- [Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity](https://mlsanity.com/paper/2510.27378/) (2025)
- [A Pragmatic Way to Measure Chain-of-Thought Monitorability](https://mlsanity.com/paper/2510.23966/) (2025)
- [Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure](https://mlsanity.com/paper/2602.08783/) (2026)
- [Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory](https://mlsanity.com/paper/2602.18297/) (2026)
- [The Illusion of Insight in Reasoning Models](https://mlsanity.com/paper/2601.00514/) (2026)
- [A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring](https://mlsanity.com/paper/2602.23163/) (2026)
- [Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs](https://mlsanity.com/paper/2602.08563/) (2026)
- [Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework](https://mlsanity.com/paper/2507.12872/) (2025)
- [State over Tokens: Characterizing the Role of Reasoning Tokens](https://mlsanity.com/paper/2512.12777/) (2025)
- [Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks](https://mlsanity.com/paper/2601.23086/) (2026)
- [Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?](https://mlsanity.com/paper/2603.30036/) (2026)
- [Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems](https://mlsanity.com/paper/2606.28425/) (2026)
- [Language Models and Logic Programs for Trustworthy Tax Reasoning](https://mlsanity.com/paper/2508.21051/) (2025)
- [Unsupervised decoding of encoded reasoning using language model interpretability](https://mlsanity.com/paper/2512.01222/) (2025)
- [Training on Documents About Monitoring Leads to CoT Obfuscation](https://mlsanity.com/paper/2605.15257/) (2026)
- [A Concrete Roadmap towards Safety Cases based on Chain-of-Thought Monitoring](https://mlsanity.com/paper/2510.19476/) (2025)
- [Diagnosing Pathological Chain-of-Thought in Reasoning Models](https://mlsanity.com/paper/2602.13904/) (2026)
- [Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations](https://mlsanity.com/paper/2607.20379/) (2026)
- [Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings](https://mlsanity.com/paper/2608.04735/) (2026)
- [Not Just the Destination, But the Journey: Reasoning Traces Causally Shape Generalization Behaviors](https://mlsanity.com/paper/2603.12397/) (2026)
- [Despite Instructions: Frontier Agents Improvise Covert Channels at Test Time](https://mlsanity.com/paper/2609.32701/) (2026)
- [NEST: Nascent Encoded Steganographic Thoughts](https://mlsanity.com/paper/2602.14095/) (2026)
- [Disentangling Deception and Hallucination Failures in LLMs](https://mlsanity.com/paper/2602.14529/) (2026)
- [Risk Reporting for Developers' Internal AI Model Use](https://mlsanity.com/paper/2604.24966/) (2026)
- [Tandem Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2606.28166/) (2026)
- [Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning](https://mlsanity.com/paper/2609.31121/) (2026)
- [Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models](https://mlsanity.com/paper/2609.36254/) (2026)
- [Correct Answers, Invalid Traces: What Verifiable Grade-School Math Reveals About Chain-of-Thought Traces](https://mlsanity.com/paper/2609.38107/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) reasoning not only enhances large language model performance but also provides critical insights into decision-making processes, marking it as a useful tool for monitoring model intent and planning. However, recent works have shown that banning the mention of a specific example of reward hacking causes obfuscation of the undesired reasoning traces but the persistence of the undesired behavior, threatening the reliability of CoT monitoring. We provide an extension to these results with regard to the ability of models to learn a specific type of obfuscated reasoning: steganography. First, we show that penalizing the use of specific strings within load-bearing reasoning traces causes models to substitute alternative strings. Crucially, this does not alter the underlying method by which the model performs the task, demonstrating that the model can learn to steganographically encode its reasoning.We further demonstrate that models can generalize an encoding scheme. When the penalized strings belong to an overarching class, the model learns not only to substitute strings seen in training, but also develops a general encoding scheme for all members of the class which it can apply to held-out testing strings.