Stealing Reasoning Traces from Proprietary LLM APIs
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 16 of its 45 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
16 of 45 references.
Closest ancestors · 3
Same direction, newest first
- 2026Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior18 Jun
- 2026PostTrainBench: Can LLM Agents Automate LLM Post-Training?9 Mar
- 2024OpenAI o1 System Card21 Dec
Other references · 13
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2025Humanity's Last Examcited by 491
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
- 2026Kimi K3: Open Frontier Intelligencecited by 138
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 90
- 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 86
- 2025Black-Box On-Policy Distillation of Large Language Modelscited by 59
- 2026ClawBench: Can AI Agents Complete Everyday Online Tasks?cited by 33
- 2026Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMscited by 17
- 2026How to Steal Reasoning Without Reasoning Tracescited by 8
1 more
- 2026Reference-Based Distillation Detection in LLMscited by 2
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Modelscited by 1
- 2026Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?cited by 0
- 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Modelscited by 0
- 2026LLM Agents Can Easily Tamper With Their Own Tracescited by 0
- 2026Distillation Defenses Easily Break After Reinforcement Learningcited by 0
- 2026On Language Drift during RLVR Post-Trainingcited by 0
- 2026Deception by Omission: Language Models Knowingly Hide Their Mistakescited by 0
- 2026Thinking Inertia: LLMs Keep Thinking When Told Not Tocited by 0
- 2026Hybrid Reasoning Agentscited by 0
- 2026Hybrid Reasoning Agents13 Oct 2026
- 2026Thinking Inertia: LLMs Keep Thinking When Told Not To8 Oct 2026
- 2026Deception by Omission: Language Models Knowingly Hide Their Mistakes8 Oct 2026
- 2026On Language Drift during RLVR Post-Training1 Oct 2026
- 2026Distillation Defenses Easily Break After Reinforcement Learning28 Sep 2026
- 2026LLM Agents Can Easily Tamper With Their Own Traces24 Sep 2026
- 2026Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models22 Sep 2026
- 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models8 Sep 2026
- 2026Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?4 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2025δ-STEAL: LLM Stealing Attack with Local Differential Privacy
- 2025DistilLock: Safeguarding LLMs from Unauthorized Knowledge Distillation on the Edge
- 2025CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges
- 2025All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Language
- 2026Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors
- 2024When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers
- 2024Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks
- 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs