mlsanityData to 14 Oct 2026

Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models

arXiv 2610.15218 ↗First posted 10 Oct 2026cs.AIcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

A recurrent-depth language model reuses one block of weights several times in a row. The number of repetitions, which we call loops, is chosen when the model runs, not when it is trained, and running fewer is cheaper and faster, so a deployment may choose to do it. Running fewer loops also makes these models less safe, and the loss of safety is larger than the loss of anything else we measured. We study four released Ouro checkpoints and Huginn-0125. At half of its trained depth, Ouro-1.4B-Thinking still solves 0.692 of the GSM8K maths problems, against 0.775 at full depth, keeping 89% of its accuracy. Over the same reduction, the rate at which it helps with harmful requests rises from 0.038 to 0.205, 5.3 times higher. The information is not lost: a linear classifier reads whether a request is harmful from the model's internal state after a single loop, with an ROC AUC of 0.967 against 0.986 at full depth. What the missing loops remove is the refusal that should follow. We measure the effect with four instruments, one generating no text at all, and reproduce it on a second model family. Removing a single direction from Ouro-1.4B's internal state at full depth drops its refusal rate on harmful prompts from 0.950 to 0.130, identifying what the shallow model is missing. Two run-time repairs do not help: the models' early-exit rules do not spend more loops on harmful requests, and choosing a depth per request does not recover the loss. Changing the training does. Standard safety training makes the shallow model safe only by refusing 0.865 of harmless requests, while training at randomly chosen depths reaches a separation of +0.399 between harmful and harmless refusal at one loop, against +0.118. On Huginn-0125 it keeps harmful help at or below 0.040 at every depth from 1 to 32 loops, where standard training reaches 0.194, and it costs no accuracy at full depth.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 60 of its 81 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

60 of 81 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Ulterior Motives: Detecting Misaligned Reasoning in Continuous Thought Models25 Apr
  2. 2025Qwen3Guard Technical Report16 Oct
  3. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  4. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  5. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
  6. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 53

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  10. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  12. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
38 more
  1. 2023Steering Language Models With Activation Engineeringcited by 922
  2. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  3. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  4. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  5. 2018Universal Transformerscited by 647
  6. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  7. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  8. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  9. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  10. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  11. 2024A StrongREJECT for Empty Jailbreakscited by 385
  12. 2022Confident Adaptive Language Modelingcited by 310
  13. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  14. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  15. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  16. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
  17. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  18. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  19. 2023Looped Transformers as Programmable Computerscited by 237
  20. 2021PonderNet: Learning to Pondercited by 168
  21. 2025Scaling Latent Reasoning via Looped Language Modelscited by 165
  22. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 149
  23. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 134
  24. 2024Looped Transformers for Length Generalizationcited by 99
  25. 2025A Survey on Latent Reasoningcited by 81
  26. 2024Exploiting LLM Quantizationcited by 65
  27. 2024Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?cited by 65
  28. 2024MoEUT: Mixture-of-Experts Universal Transformerscited by 59
  29. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 57
  30. 2024On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encodingcited by 36
  31. 2023Sparse Universal Transformercited by 32
  32. 2024Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuningcited by 26
  33. 2025Think-at-Hard: Dynamic Looped Transformers for Improved Reasoningcited by 16
  34. 2025Dr.LLM: Dynamic Layer Routing in LLMscited by 12
  35. 2026Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readoutscited by 12
  36. 2025To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformerscited by 11
  37. 2026Steering Recurrent Reasoners at Inference Time with Readout Feedbackcited by 1
  38. 2026Assessing Adversarial Robustness of Latent Reasoning Modelscited by 1

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models
  2. 2026The Geometry of Refusal: Why Post-Hoc Safety Is Fragile and Pretraining-Time Safety Persists
  3. 2025Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint
  4. 2024Steering Language Model Refusal with Sparse Autoencoders
  5. 2026LLMs Can Unlearn Refusal with Only 1,000 Benign Samples
  6. 2026Refusal Localizes, the Damage Relocates: Safety Layers Under Few-Sample Fine-Tuning
  7. 2026Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries
  8. 2026CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models