mlsanityData to 7 Oct 2026

Knowledge Distillation Detection for Open-weights Models

arXiv 2510.02302 ↗First posted 2 Oct 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose the task of knowledge distillation detection, which aims to determine whether a student model has been distilled from a given teacher, under a practical setting where only the student's weights and the teacher's API are available. This problem is motivated by growing concerns about model provenance and unauthorized replication through distillation. To address this task, we introduce a model-agnostic framework that combines data-free input synthesis and statistical score computation for detecting distillation. Our approach is applicable to both classification and generative models. Experiments on diverse architectures for image classification and text-to-image generation show that our method improves detection accuracy over the strongest baselines by 59.6% on CIFAR-10, 71.2% on ImageNet, and 20.0% for text-to-image generation. The code is available at https://github.com/shqii1j/distillation_detection.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 51 of its 65 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

51 of 65 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  11. 2023Consistency Modelscited by 1,778
  12. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
38 more
  1. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  2. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  3. 2022Red Teaming Language Models with Language Modelscited by 983
  4. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  5. 2019Relational Knowledge Distillationcited by 873
  6. 2021Membership Inference Attacks From First Principlescited by 839
  7. 2023Extracting Training Data from Diffusion Modelscited by 723
  8. 2022On Distillation of Guided Diffusion Modelscited by 721
  9. 2023A Watermark for Large Language Modelscited by 667
  10. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  11. 2019Similarity-Preserving Knowledge Distillationcited by 507
  12. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  13. 2017Deep Layer Aggregationcited by 436
  14. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  15. 2022Decoupled Knowledge Distillationcited by 380
  16. 2020Label-Only Membership Inference Attackscited by 371
  17. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  18. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  19. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  20. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  21. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  22. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
  23. 2024Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesiscited by 153
  24. 2022Knowledge Distillation with the Reused Teacher Classifiercited by 140
  25. 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
  26. 2017Dual Path Networkscited by 119
  27. 2024PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Acceleratorcited by 95
  28. 2020Generative Low-bitwidth Data Free Quantizationcited by 87
  29. 2023One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillationcited by 77
  30. 2017DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfercited by 55
  31. 2021Qimera: Data-free Quantization with Synthetic Boundary Supporting Samplescited by 39
  32. 2022Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replaycited by 35
  33. 2023MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splittingcited by 33
  34. 2020MixMix: All You Need for Data-Free Compression Are Feature and Data Mixingcited by 29
  35. 2023Adaptive Data-Free Quantizationcited by 26
  36. 2023IMMA: Immunizing text-to-image Models against Malicious Adaptationcited by 17
  37. 2025Model Immunization from a Condition Number Perspectivecited by 8
  38. 2024Learning to Obstruct Few-Shot Image Classification over Restricted Classescited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Reference-Based Distillation Detection in LLMscited by 2
  2. 2026TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenancecited by 1
  3. 2026Provable Model Provenance Set for Large Language Modelscited by 0
  4. 2026Unknown-Traffic Detection, Calibration and Shortcut Reliance in Distilled Encrypted-Traffic Classifiers over One Yearcited by 0