Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
Proprietary large language models (LLMs) entail substantial intellectual and financial investment, making them valuable intellectual property (IP). However, even when deployed via black-box APIs, these models remain vulnerable to unauthorized knowledge distillation, which allows adversaries to cheaply extract and replicate model capabilities. To address this issue, anti-distillation (AD) has been proposed to generate defensive outputs that hinder distillation effectiveness, overcoming the limitation of watermarking-based approaches that rely on post-hoc verification. However, existing AD methods based on internal model perturbations struggle to balance anti-distillability and utility (e.g., answer accuracy and naturalness) of reasoning traces, with stronger defenses often causing significant utility loss. To fill this gap, we propose textbfunderlineSkeleton-textbfunderlineGuided textbfunderlineReasoning textbfunderlineEditing (SGRE), an textitAnswer-then-Edit framework that performs post-hoc trace modification for anti-distillation. In the answer stage, the teacher model first generates clean reasoning traces, preserving the original reasoning accuracy while enabling more flexible control over trace naturalness. In the editing stage, we draw inspiration from Cognitive Load Theory (CLT) and introduce a three-stage strategy consisting of reasoning skeleton extraction, skeleton graph coarsening, and skeleton verbalization. These operations jointly perturb reasoning structures and augment textual complexity to amplify extraneous load on student models, hindering their acquisition of underlying reasoning patterns. Extensive experiments across diverse LLMs demonstrate that SGRE achieves state-of-the-art performance in reducing distillation effectiveness, while maintaining lossless reasoning accuracy and superior trace naturalness.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 43 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
29 of 43 references.
Closest ancestors · 1
Same direction, newest first
Other references · 28
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2025Qwen3 Technical Reportcited by 8,195
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2025Gemma 3 Technical Reportcited by 1,686
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2025OpenAI GPT-5 System Cardcited by 910
16 more
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2016Sequence-Level Knowledge Distillationcited by 761
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
- 2023A Watermark for Large Language Modelscited by 667
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2023Poisoning Language Models During Instruction Tuningcited by 254
- 2023f-Divergence Minimization for Sequence-Level Knowledge Distillationcited by 112
- 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermarkcited by 78
- 2024Instructional Fingerprinting of Large Language Modelscited by 64
- 2025Do Large Language Model Benchmarks Test Reliability?cited by 50
- 2022Watermarking Pre-trained Language Models with Backdooringcited by 44
- 2025Improve LLM-as-a-Judge Ability as a General Abilitycited by 35
- 2022Distillation-Resistant Watermarking for Model Protection in NLPcited by 29
- 2025Antidistillation Samplingcited by 24
- 2025ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via α-β-Divergencecited by 21
- 2025DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillationcited by 14
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
- 2026Stealing Reasoning Traces from Proprietary LLM APIs
- 2025Information-Preserving Reformulation of Reasoning Traces for Antidistillation
- 2026Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs
- 2025Not All Correct Answers Are Equal: Why Your Distillation Source Matters
- 2026DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation
- 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps
- 2026Negative Self-Distillation: Learning to Reason by Avoiding Flaws