mlsanityData to 7 Oct 2026

Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets

arXiv 2607.08173 ↗First posted 9 Jul 2026cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Black box auditing of language models is an essential pre-deployment tool, but it may miss subtle forms of misalignment and hidden information. To better elicit hidden information during an auditing process, we introduce overthinking: the process of using reasoning task vectors to amplify the propensity to think out loud of reasoning models. Given the parameters of a non-reasoning instruct model M and reasoning-distilled model R, we define the overthinking model as boldsymbolθOα = boldsymbolθM + α(boldsymbolθR - boldsymbolθM), where α > 1 amplifies reasoning beyond the pure reasoning model R. Additionally, we introduce new layer-wise attenuation strategies that selectively amplify reasoning without losing quality and coherence of model outputs. We demonstrate that overthinking models are more likely to reveal hidden information across four experimental settings, across 2B-32B models. Our findings suggest that reasoning amplification may surface secrets or unintended behaviors acquired during training up to 10× more frequently than the original reasoning model. How secrets surface depends on the secret type: some require perturbation along the reasoning direction, while others yield to any sufficiently large weight perturbation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 29 of its 39 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

29 of 39 references.

References · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  7. 2022Editing Models with Task Arithmeticcited by 1,076
  8. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  9. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  10. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  12. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
17 more
  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  2. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  3. 2024Alignment faking in large language modelscited by 316
  4. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 248
  5. 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 218
  6. 2023Model evaluation for extreme riskscited by 167
  7. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 166
  8. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  9. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 69
  10. 2024LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Mergingcited by 39
  11. 2025Eliciting Secret Knowledge from Language Modelscited by 23
  12. 2024Noise Injection Reveals Hidden Capabilities of Sandbagging Language Modelscited by 22
  13. 2025Steering Language Models with Weight Arithmeticcited by 19
  14. 2024Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internalscited by 13
  15. 2025Dynamic Fisher-weighted Model Merging via Bayesian Optimizationcited by 12
  16. 2025Prefill-level Jailbreak: A Black-Box Risk Analysis of Large Language Modelscited by 9
  17. 2025Noise Injection Systemically Degrades Large Language Model Safety Guardrailscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Abilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring
  2. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models
  3. 2025Audit-of-Understanding: Posterior-Constrained Inference for Mathematical Reasoning in Language Models
  4. 2026The Illusion of Reasoning: Exposing Evasive Data Contamination in LLMs via Zero-CoT Truncation
  5. 2025OverThink: Slowdown Attacks on Reasoning LLMs
  6. 2026RAudit: A Blind Auditing Protocol for Large Language Model Reasoning
  7. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  8. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steering