Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
also in 1 more
Abstract (arXiv)
Distillation via reasoning traces exposes frontier models to adversarial third parties, who can bypass their guardrails and misappropriate their capabilities. Existing antidistillation defenses perturb the teacher's sampling distribution to hinder downstream student learning, but these perturbations also degrade the trace quality even when task accuracy is preserved, often leaving visible artifacts such as syntactic incoherencies and repetition chains. Beyond eroding user trust, such traces are statistically distinguishable, allowing adversaries to filter or resample them and weaken the defense. An effective defense must therefore limit how visibly it degrades the teacher. To address this gap, we formulate antidistillation as a max-min game whose constraint set explicitly encodes defense detectability. Satisfying both axes--- degrading the student while remaining undetectable---may require more clever perturbations, akin to sparse but targeted adversarial attacks in robustness literature. We apply the same analogy to reasoning steps, targeting thought anchors: sentences identified in prior interpretability work as strongly influencing a model's output. We propose Thought Anchor RemovalS (TARS), which identifies and removes thought anchors, whose content is semantically redundant, so that the reasoning trace remains legible to the user. Modeling these removals as burst-deletions, we then provide formal guarantees on its detectability. Finally, we empirically demonstrate the efficacy of TARS in consistently hindering downstream distillation across teacher model scales (DeepSeek R1 Distill Qwen 7B, Gemma 3 12B), reasoning benchmarks (MMLU Pro, MATH, DeepMath), and student architectures (Llama 3.2 1B, Llama 3.2 3B, Gemma 3 1B, Qwen 3 1.7B) while preserving teacher performance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 48 references that are in the map. First posted on arXiv 25 Apr 2026.
Builds on
16 of 48 references.
Closest ancestors · 2
Same direction, newest first
- 2025Information-Preserving Reformulation of Reasoning Traces for Antidistillation13 Oct
- 2025Thought Anchors: Which LLM Reasoning Steps Matter?23 Jun
Other references · 14
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2025s1: Simple test-time scalingcited by 1,341
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2017One pixel attack for fooling deep neural networkscited by 582
- 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 72
- 2018Maximal Jacobian-based Saliency Map Attackcited by 46
- 2025Antidistillation Samplingcited by 25
- 2025DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillationcited by 14
- 2026TRIM: Hybrid Inference via Targeted Stepwise Routing in Multi-Step Reasoning Taskscited by 8
- 2026What Matters For Safety Alignment?cited by 7
- 2026Protecting Language Models Against Unauthorized Distillation through Trace Rewritingcited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TraceGuard: Process-Guided Firewall against Reasoning Backdoors in Large Language Models
- 2025In Their Own Words: Reasoning Traces Tailored for Small Models Make Them Better Reasoners
- 2025Reasoning Models Sometimes Output Illegible Chains of Thought
- 2026Evading Chain-of-Thought Monitoring Through Model Poisoning
- 2025ExplainableGuard: Interpretable Adversarial Defense for Large Language Models Using Chain-of-Thought Reasoning
- 2025Large language models can learn and generalize steganographic chain-of-thought under process supervision
- 2026Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs
- 2026The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages