mlsanityData to 14 Oct 2026

Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries

arXiv 2605.18891 ↗First posted 17 May 2026cs.LG cs.AIcited by 7

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Evaluations of unlearning on reasoning models sometimes show a bypass pattern. The answer side looks unlearned, but the model's own thinking trace keeps emitting the forgotten content, and the gap is taken as evidence that the weights still remember. We audit this reading on DeepSeek-R1-Distill-Qwen-7B with LoRA-memorized fictional authors and NPO unlearning, conditioned on a six-token canary head. On one seed, swapping the thinking trace for a short non-canary prefill on the same weights drops the answer rate by as much as the bypass gap itself, whether the prefill mimics the training template or not. On a second seed the bypass gap shrinks rather than vanishing, and the prefill swap reverses direction and brings the answer rate to ceiling. A positive parser-split bypass gap thus does not by itself identify hidden weight-level memorization, and does not rule it out either. On a different distillate the same metric flips sign because the parser cannot find the closing tag. We recommend a decode-time template swap as a cheap sanity check alongside the canonical audit.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 24 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

20 of 24 references.

References · 20

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  5. 2019Machine Unlearningcited by 1,061
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  7. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  9. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  10. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  12. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
8 more
  1. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
  2. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 381
  3. 2019Measurement and Fairnesscited by 308
  4. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 163
  5. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 158
  6. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  7. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
  8. 2025Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Selective Forgetting for Large Reasoning Models
  2. 2026Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets
  3. 2026GUARD: Natural Forgetting in Large Reasoning Models via Guided Answer-Reasoning Distillation
  4. 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning
  5. 2025Interpretable Traces, Unexpected Outcomes: Investigating the Disconnect in Trace-Based Knowledge Distillation
  6. 2026Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs
  7. 2026How Much Does a Reasoning Summary Reveal? An Observability Ladder for Large Language Models
  8. 2025In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners