mlsanityData to 14 Oct 2026

Antidistillation Sampling

arXiv 2504.13146 ↗First posted 17 Apr 2025cs.AI cs.CLcited by 25

Not on the map. Why?

Abstract (arXiv)

Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. Antidistillation sampling provides exactly this capability. By strategically modifying a model's next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model's practical utility. For further details, see https://antidistillation.com.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 17 Apr 2025.

Builds on

38 of 55 references.

References · 38

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2024Qwen2 Technical Reportcited by 2,120
  11. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  12. 2020Knowledge Distillation: A Surveycited by 1,603
26 more
  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  3. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  4. 2023A Watermark for Large Language Modelscited by 725
  5. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  6. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  7. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  8. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
  9. 2018Model compression via distillation and quantizationcited by 289
  10. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 180
  11. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  12. 2024Stealing Part of a Production Language Modelcited by 159
  13. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 158
  14. 2022Learning by Distilling Contextcited by 138
  15. 2023Controlled Decoding from Language Modelscited by 135
  16. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 128
  17. 2019Adversarially Robust Distillationcited by 125
  18. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 90
  19. 2023On the Learnability of Watermarks for Language Modelscited by 81
  20. 2024Watermarking Makes Language Models Radioactivecited by 58
  21. 2025Do Large Language Model Benchmarks Test Reliability?cited by 54
  22. 2021Undistillable: Making A Nasty Teacher That CANNOT teach studentscited by 36
  23. 2025Adversarial Reasoning at Jailbreaking Timecited by 29
  24. 2024QUEEN: Query Unlearning against Model Extractioncited by 9
  25. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  26. 2024Distilling Algorithmic Reasoning from LLMs via Explaining Solution Programscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 17.

25 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  2. 2025DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillationcited by 14
  3. 2025Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Servicescited by 10
  4. 2025Model Immunization from a Condition Number Perspectivecited by 8
  5. 2025Information-Preserving Reformulation of Reasoning Traces for Antidistillationcited by 8
  6. 2026Antidistillation Fingerprintingcited by 6
  7. 2026Protecting Language Models Against Unauthorized Distillation through Trace Rewritingcited by 6
  8. 2025In Good GRACEs: Principled Teacher Selection for Knowledge Distillationcited by 5
  9. 2026Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspectivecited by 5
  10. 2025Evaluating Language Model Reasoning about Confidential Informationcited by 3
  11. 2026RedAct: Redacting Agent Capability Traces for Procedural Skill Protectioncited by 3
  12. 2026Asking Back: Interaction-Layer Antidistillation Watermarkscited by 2
13 more
  1. 2026Lossless Anti-Distillation Samplingcited by 1
  2. 2026What Does It Mean to Break a Distillation Defense?cited by 1
  3. 2025Conscious Data Contribution via Community-Driven Chain-of-Thought Distillationcited by 0
  4. 2026Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Modelscited by 0
  5. 2026The Distillation Game: Adaptive Evaluations & Efficient Defensescited by 0
  6. 2026ADS-C: Antidistillation Sampling for Classificationcited by 0
  7. 2026Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utilitycited by 0
  8. 2026Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuningcited by 0
  9. 2026Caliber: Cross-Architecture Extraction-Cost Control for Score-Returning APIscited by 0
  10. 2026Spectral Saliency for Machine Unlearningcited by 0
  11. 2026Retrospective Distillation Attribution via Normalized Response Similaritycited by 0
  12. 2026Distillation Defenses Easily Break After Reinforcement Learningcited by 0
  13. 2026Radioactive Feedback: Tracing Models Trained by Label-Only LLM Supervisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Poster: A Preliminary Study of LLM Distillation Inference
  2. 2025Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
  3. 2025Detecting Distillation Data from Reasoning Models
  4. 2026When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling
  5. 2026Reasoning with Sampling: Cutting at Decision Points
  6. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Think
  7. 2026Reference-Based Distillation Detection in LLMs
  8. 2026Does self-distillation improve reasoning?