# Defense against Unauthorized Distillation in Image Restoration via Feature Space Perturbation

arXiv: https://arxiv.org/abs/2510.08925 · HTML: https://mlsanity.com/paper/2510.08925/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 32 of its 64 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 32)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Anti-Backdoor Learning: Training Clean Models on Poisoned Data](https://mlsanity.com/paper/2110.11571/) (2021)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method](https://mlsanity.com/paper/2212.11548/) (2022)
- [Data-Free Knowledge Distillation for Deep Neural Networks](https://mlsanity.com/paper/1710.07535/) (2017)
- [Residual Feature Distillation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2009.11551/) (2020)
- [Fast and Accurate Single Image Super-Resolution via Information Distillation Network](https://mlsanity.com/paper/1803.09454/) (2018)
- [U-shape Transformer for Underwater Image Enhancement](https://mlsanity.com/paper/2111.11843/) (2021)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [[Extended version] Rethinking Deep Neural Network Ownership Verification: Embedding Passports to Defeat Ambiguity Attacks](https://mlsanity.com/paper/1909.07830/) (2019)
- [GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions](https://mlsanity.com/paper/2305.17863/) (2023)
- [Backdoor Attacks against Transfer Learning with Pre-trained Deep Learning Models](https://mlsanity.com/paper/2001.03274/) (2020)
- [Undistillable: Making A Nasty Teacher That CANNOT teach students](https://mlsanity.com/paper/2105.07381/) (2021)
- [MB-TaylorFormer V2: Improved Multi-branch Linear Transformer Expanded by Taylor Formula for Image Restoration](https://mlsanity.com/paper/2501.04486/) (2025)
- [StegaNeRF: Embedding Invisible Information within Neural Radiance Fields](https://mlsanity.com/paper/2212.01602/) (2022)
- [Knowledge Distillation in Deep Learning and its Applications](https://mlsanity.com/paper/2007.09029/) (2020)
- [MC-Blur: A Comprehensive Benchmark for Image Deblurring](https://mlsanity.com/paper/2112.00234/) (2021)
- [Teacher-Student Architecture for Knowledge Learning: A Survey](https://mlsanity.com/paper/2210.17332/) (2022)
- [Knowledge Distillation in Vision Transformers: A Critical Review](https://mlsanity.com/paper/2302.02108/) (2023)
- [Generator Versus Segmentor: Pseudo-healthy Synthesis](https://mlsanity.com/paper/2009.05722/) (2020)
- [Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration](https://mlsanity.com/paper/2203.08444/) (2022)
- [KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow](https://mlsanity.com/paper/2004.05319/) (2020)
- [Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration](https://mlsanity.com/paper/2412.08939/) (2024)
- [Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation](https://mlsanity.com/paper/2204.13873/) (2022)
- [ConcealGS: Concealing Invisible Copyright Information in 3D Gaussian Splatting](https://mlsanity.com/paper/2501.03605/) (2025)
- [Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples](https://mlsanity.com/paper/2310.12707/) (2023)

## Abstract (arXiv)

Knowledge distillation (KD) attacks pose a significant threat to deep model intellectual property by enabling adversaries to train student networks using a teacher model's outputs. While recent defenses in image classification have successfully disrupted KD by perturbing output probabilities, extending these methods to image restoration is difficult. Unlike classification, restoration is a generative task with continuous, high-dimensional outputs that depend on spatial coherence and fine details. Minor perturbations are often insufficient, as students can still learn the underlying mapping.To address this, we propose Adaptive Singular Value Perturbation (ASVP), a runtime defense tailored for image restoration models. ASVP operates on internal feature maps of the teacher using singular value decomposition (SVD). It amplifies the topk singular values to inject structured, high-frequency perturbations, disrupting the alignment needed for distillation. This hinders student learning while preserving the teacher's output quality.We evaluate ASVP across five image restoration tasks: super-resolution, low-light enhancement, underwater enhancement, dehazing, and deraining. Experiments show ASVP reduces student PSNR by up to 4 dB and SSIM by 60-75%, with negligible impact on the teacher's performance. Compared to prior methods, ASVP offers a stronger and more consistent defense.Our approach provides a practical solution to protect open-source restoration models from unauthorized knowledge distillation.