mlsanityData to 7 Oct 2026

GUARD: Natural Forgetting in Large Reasoning Models via Guided Answer-Reasoning Distillation

arXiv 2609.21677 ↗First posted 18 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking also fits: Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Recent advances in large reasoning models (LRMs) have made machine unlearning more challenging, as protected facts or unsafe rationales may surface in intermediate chain-of-thought (CoT) traces before the final answer is produced. Existing unlearning objectives typically suppress the target content or redirect internal representations, but they never specify how the post-forgetting trajectory should continue, which can lead to hallucinated substitutes, malformed boundaries, or repetitive outputs. We argue that LRM unlearning should instead learn a natural forgetting trajectory: a coherent non-disclosing CoT followed by a stable refusal-style answer that replace the original disclosure. To this end, we propose Guided Answer-Reasoning Distillation (GUARD), which converts model-generated unsafe disclosures into safe-exit trajectories, aligns a frozen LRM via guidance tokens, and distills the guided behavior into model parameters. To address the lack of metrics for replacement quality beyond leakage, we further introduce Natural Forgetting Reasoning Score (NFRS), which captures structural stability, fluency, and unsupported substitutes in forgotten outputs. Extensive experiments on R-TOFU and a STAR-1-derived harmful-intent setting show that GUARD substantially reduces unsafe and privacy disclosures across two widely adopted distilled LRMs while preserving reasoning utility.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 8 of its 33 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

8 of 33 references.

References · 8

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  8. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 67

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Selective Forgetting for Large Reasoning Models
  2. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills
  3. 2025Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models
  4. 2026Masked Distillation: Internalizing the Chain-of-Thought in Language Models
  5. 2026Free(): Learning to Forget in Malloc-Only Reasoning Models
  6. 2025R-TOFU: Unlearning in Large Reasoning Models
  7. 2026Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
  8. 2026Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Paths