A Public Theory of Distillation Resistance via Constraint-Coupled Reasoning Architectures
On the map: Knowledge distillation → On-policy and self-distillation for LLMs
Abstract (arXiv)
Knowledge distillation, model extraction, and behavior transfer have become central concerns in frontier AI. The main risk is not merely copying, but the possibility that useful capability can be transferred more cheaply than the governance structure that originally accompanied it. This paper presents a public, trade-secret-safe theoretical framework for reducing that asymmetry at the architectural level. The core claim is that distillation becomes less valuable as a shortcut when high-level capability is coupled to internal stability constraints that shape state transitions over time. To formalize this idea, the paper introduces a constraint-coupled reasoning framework with four elements: bounded transition burden, path-load accumulation, dynamically evolving feasible regions, and a capability-stability coupling condition. The paper is intentionally public-safe: it omits proprietary implementation details, training recipes, thresholds, hidden-state instrumentation, deployment procedures, and confidential system design choices. The contribution is therefore theoretical rather than operational. It offers a falsifiable architectural thesis, a clear threat model, and a set of experimentally testable hypotheses for future work on distillation resistance, alignment, and model governance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 11 of its 14 references that are in the map. First posted on arXiv 26 Mar 2026.
Builds on
11 of 14 references.
References · 11
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2020Knowledge Distillation: A Surveycited by 1,603
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
- 2023A Watermark for Large Language Modelscited by 725
- 2016Stealing Machine Learning Models via Prediction APIscited by 619
- 2018Born Again Neural Networkscited by 546
- 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses
- 2026Distillation Defenses Easily Break After Reinforcement Learning
- 2026Institutional AI: A Governance Framework for Distributional AGI Safety
- 2026PHF: Privileged Hidden Flow for On-Policy Self-Distillation
- 2026SOD: Step-wise On-policy Distillation for Small Language Model Agents
- 2026Trust Region Policy Distillation
- 2026Constitutional On-Policy Safe Distillation
- 2025Agentic Physical AI toward a Domain-Specific Foundation Model for Energy Systems: A Case Study on Nuclear Reactor Control