# Antidistillation Sampling

arXiv: https://arxiv.org/abs/2504.13146 · HTML: https://mlsanity.com/paper/2504.13146/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 55 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 30 of 38)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [A Survey of Model Compression and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/1710.09282/) (2017)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses](https://mlsanity.com/paper/2012.10544/) (2020)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [Distilling Reasoning Capabilities into Smaller Language Models](https://mlsanity.com/paper/2212.00193/) (2022)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [Learning by Distilling Context](https://mlsanity.com/paper/2209.15189/) (2022)
- [Controlled Decoding from Language Models](https://mlsanity.com/paper/2310.17022/) (2023)
- [Universal Jailbreak Backdoors from Poisoned Human Feedback](https://mlsanity.com/paper/2311.14455/) (2023)
- [Adversarially Robust Distillation](https://mlsanity.com/paper/1905.09747/) (2019)
- [Protecting Language Generation Models via Invisible Watermarking](https://mlsanity.com/paper/2302.03162/) (2023)

## Built on by (most cited first, 25 of 25)

- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation](https://mlsanity.com/paper/2505.19504/) (2025)
- [Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services](https://mlsanity.com/paper/2505.18471/) (2025)
- [Model Immunization from a Condition Number Perspective](https://mlsanity.com/paper/2505.23760/) (2025)
- [Information-Preserving Reformulation of Reasoning Traces for Antidistillation](https://mlsanity.com/paper/2510.11545/) (2025)
- [In Good GRACEs: Principled Teacher Selection for Knowledge Distillation](https://mlsanity.com/paper/2511.02833/) (2025)
- [Antidistillation Fingerprinting](https://mlsanity.com/paper/2602.03812/) (2026)
- [Protecting Language Models Against Unauthorized Distillation through Trace Rewriting](https://mlsanity.com/paper/2602.15143/) (2026)
- [Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective](https://mlsanity.com/paper/2602.03396/) (2026)
- [Evaluating Language Model Reasoning about Confidential Information](https://mlsanity.com/paper/2508.19980/) (2025)
- [RedAct: Redacting Agent Capability Traces for Procedural Skill Protection](https://mlsanity.com/paper/2606.10813/) (2026)
- [Asking Back: Interaction-Layer Antidistillation Watermarks](https://mlsanity.com/paper/2605.16462/) (2026)
- [Lossless Anti-Distillation Sampling](https://mlsanity.com/paper/2605.18829/) (2026)
- [What Does It Mean to Break a Distillation Defense?](https://mlsanity.com/paper/2606.25059/) (2026)
- [Conscious Data Contribution via Community-Driven Chain-of-Thought Distillation](https://mlsanity.com/paper/2512.18174/) (2025)
- [Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models](https://mlsanity.com/paper/2604.23238/) (2026)
- [The Distillation Game: Adaptive Evaluations & Efficient Defenses](https://mlsanity.com/paper/2605.22737/) (2026)
- [ADS-C: Antidistillation Sampling for Classification](https://mlsanity.com/paper/2607.15467/) (2026)
- [Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility](https://mlsanity.com/paper/2607.20440/) (2026)
- [Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuning](https://mlsanity.com/paper/2607.26358/) (2026)
- [Caliber: Cross-Architecture Extraction-Cost Control for Score-Returning APIs](https://mlsanity.com/paper/2608.01023/) (2026)
- [Spectral Saliency for Machine Unlearning](https://mlsanity.com/paper/2608.15548/) (2026)
- [Retrospective Distillation Attribution via Normalized Response Similarity](https://mlsanity.com/paper/2609.32749/) (2026)
- [Distillation Defenses Easily Break After Reinforcement Learning](https://mlsanity.com/paper/2609.35699/) (2026)
- [Radioactive Feedback: Tracing Models Trained by Label-Only LLM Supervision](https://mlsanity.com/paper/2610.15685/) (2026)

## Abstract (arXiv)

Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. Antidistillation sampling provides exactly this capability. By strategically modifying a model's next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model's practical utility. For further details, see https://antidistillation.com.