mlsanityData to 7 Oct 2026

Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior

arXiv 2607.15286 ↗First posted 18 Jun 2026cs.CR cs.CL cs.LGcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

We investigate whether harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behaviour and be distilled into reusable jailbreak attacks. Using an emergent-misalignment organism and a refusal-ablated jailbroken organism, we transplant harmful CoTs into 29 open-source and 5 closed-source targets. Transferred traces raise harmful-response rates above 80% on the most vulnerable open-source models, while semantically mismatched CoTs fail entirely. LLooM concept mining identifies four recurring components of harmful reasoning: proceduralisation, ethical decoupling, evasion, and target--vulnerability framing. Distilling these patterns into reusable system prompts produces effective black-box jailbreaks, outperforming direct CoT transplantation on strongly aligned models by up to an order of magnitude, including a 10× improvement on GPT-4.1 AdvBench. Reasoning-enabled models are more than twice as vulnerable, and output-side safeguards such as Llama-Guard~3 frequently miss harmful generations. Our results show that harmful reasoning transfers at both the trace and pattern levels, motivating defences that evaluate reasoning context in addition to final outputs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 3 of its 27 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

3 of 27 references.

References · 3

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  3. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models
  2. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models
  3. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
  4. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attack
  5. 2025Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption
  6. 2025Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking?
  7. 2026Evading Chain-of-Thought Monitoring Through Model Poisoning
  8. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking