Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
On the map: LLM reasoning → Test-time compute and efficient thinking
also in 1 more
Abstract (arXiv)
Evaluations of unlearning on reasoning models sometimes show a bypass pattern. The answer side looks unlearned, but the model's own thinking trace keeps emitting the forgotten content, and the gap is taken as evidence that the weights still remember. We audit this reading on DeepSeek-R1-Distill-Qwen-7B with LoRA-memorized fictional authors and NPO unlearning, conditioned on a six-token canary head. On one seed, swapping the thinking trace for a short non-canary prefill on the same weights drops the answer rate by as much as the bypass gap itself, whether the prefill mimics the training template or not. On a second seed the bypass gap shrinks rather than vanishing, and the prefill swap reverses direction and brings the answer rate to ceiling. A positive parser-split bypass gap thus does not by itself identify hidden weight-level memorization, and does not rule it out either. On a different distillate the same metric flips sign because the parser cannot find the closing tag. We recommend a decode-time template swap as a cheap sanity check alongside the canonical audit.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 24 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
20 of 24 references.
References · 20
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2019Machine Unlearningcited by 1,061
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
- 2022Quantifying Memorization Across Neural Language Modelscited by 763
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
- 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
- 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
8 more
- 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
- 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 381
- 2019Measurement and Fairnesscited by 308
- 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 163
- 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 158
- 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
- 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
- 2025Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Modelscited by 6
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agentscited by 7
- 2026Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MMEcited by 1
- 2026Auditing the Audit: Five Failure Modes in Benchmark-Validity Auditscited by 1
- 2026Probe Choice Changes Canary-Memorization Verdicts: Three Post-Hoc Disagreement Case Studies in a Text-Dominant LoRA-Tuned Autoregressive Testbedcited by 0
- 2026Where Privacy Risk Lives in English-Source Multilingual RAG: A Stage-Decomposed Audit Across Five Query Languagescited by 0
- 2026Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generationcited by 0
- 2026Exploratory As-Analyzed No-Detection of Culturally-Marked Predicate-Triggered PII Amplification in a Synthetic-English RAG Probe: A Predicate-Resource-Confounded Auditcited by 0
- 2026Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generation10 Aug 2026
- 2026Auditing the Audit: Five Failure Modes in Benchmark-Validity Audits1 Jul 2026
- 2026Probe Choice Changes Canary-Memorization Verdicts: Three Post-Hoc Disagreement Case Studies in a Text-Dominant LoRA-Tuned Autoregressive Testbed30 Jun 2026
- 2026When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents22 Jun 2026
- 2026Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MME22 Jun 2026
- 2026Exploratory As-Analyzed No-Detection of Culturally-Marked Predicate-Triggered PII Amplification in a Synthetic-English RAG Probe: A Predicate-Resource-Confounded Audit16 Jun 2026
- 2026Where Privacy Risk Lives in English-Source Multilingual RAG: A Stage-Decomposed Audit Across Five Query Languages26 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Selective Forgetting for Large Reasoning Models
- 2026Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets
- 2026GUARD: Natural Forgetting in Large Reasoning Models via Guided Answer-Reasoning Distillation
- 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning
- 2025Interpretable Traces, Unexpected Outcomes: Investigating the Disconnect in Trace-Based Knowledge Distillation
- 2026Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs
- 2026How Much Does a Reasoning Summary Reveal? An Observability Ladder for Large Language Models
- 2025In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners