# SPARK: Spatial Policy-driven Adaptive Reinforcement learning for Knowledge distillation

arXiv: https://arxiv.org/abs/2606.15243 · HTML: https://mlsanity.com/paper/2606.15243/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 13 Jun 2026.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)
- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Learning Enriched Features for Fast Image Restoration and Enhancement](https://mlsanity.com/paper/2205.01649/) (2022)
- [Residual Local Feature Network for Efficient Super-Resolution](https://mlsanity.com/paper/2205.07514/) (2022)
- [Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning](https://mlsanity.com/paper/1804.03312/) (2018)
- [Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2108.11510/) (2021)
- [DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks](https://mlsanity.com/paper/2012.11230/) (2020)
- [ReLLIE: Deep Reinforcement Learning for Customized Low-Light Image Enhancement](https://mlsanity.com/paper/2107.05830/) (2021)
- [The Tenth NTIRE 2025 Image Denoising Challenge Report](https://mlsanity.com/paper/2504.12276/) (2025)
- [FAKD: Feature Augmented Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2208.14143/) (2022)
- [Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration](https://mlsanity.com/paper/2412.08939/) (2024)
- [Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution](https://mlsanity.com/paper/2404.02573/) (2024)
- [Knowledge Distillation for Image Restoration : Simultaneous Learning from Degraded and Clean Images](https://mlsanity.com/paper/2501.09268/) (2025)
- [Decoder-Free Distillation for Quantized Image Restoration](https://mlsanity.com/paper/2603.09624/) (2026)

## Abstract (arXiv)

Low-bit quantization enables deployment of image restoration (IR) networks on resource-constrained devices, but introduces rounding noise that disproportionately degrades high-frequency regions such as edges and fine textures. Existing knowledge distillation (KD) methods apply distillation signals uniformly across all spatial locations, overlooking the varying reconstruction difficulty across image regions. To address this, we propose SPARK (Spatial Policy-driven Adaptive Reinforcement Learning for Knowledge Distillation), a framework that adaptively allocates distillation effort using a lightweight reinforcement learning (RL) policy network. At each training step, a difficulty feature extractor computes four signals, namely Laplacian variance, pixel variance, student reconstruction error, and teacher-student knowledge gap, which are fed into a compact policy CNN that produces a stochastic spatial weight map to modulate the KD loss during quantization-aware training (QAT). SPARK is IR task-agnostic, adds no inference cost, and integrates into any existing QAT pipeline without architectural changes. Experiments on benchmark datasets demonstrate that SPARK consistently outperforms PTQ, QAT, and state-of-the-art (SOTA) KD approaches across multiple student architectures, achieving reconstruction quality closest to the full-precision teacher under significant computational constraints.