# Adapting Models to Signal Degradation using Distillation

arXiv: https://arxiv.org/abs/1604.00433 · HTML: https://mlsanity.com/paper/1604.00433/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2016.

## Built on by (most cited first, 11 of 11)

- [Object DGCNN: 3D Object Detection using Dynamic Graphs](https://mlsanity.com/paper/2110.06923/) (2021)
- [A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification](https://mlsanity.com/paper/2104.00679/) (2021)
- [The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshop](https://mlsanity.com/paper/2103.06937/) (2021)
- [Exploiting Diffusion Prior for Task-driven Image Restoration](https://mlsanity.com/paper/2507.22459/) (2025)
- [Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities](https://mlsanity.com/paper/2403.04908/) (2024)
- [Efficient Low-Resolution Face Recognition via Bridge Distillation](https://mlsanity.com/paper/2409.11786/) (2024)
- [Total-Body Low-Dose CT Image Denoising using Prior Knowledge Transfer Technique with Contrastive Regularization Mechanism](https://mlsanity.com/paper/2112.00729/) (2021)
- [Look Through Masks: Towards Masked Face Recognition with De-Occlusion Distillation](https://mlsanity.com/paper/2409.12385/) (2024)
- [Teacher Network Calibration Improves Cross-Quality Knowledge Distillation](https://mlsanity.com/paper/2304.07593/) (2023)
- [Addressing the Accuracy-Cost Tradeoff in Material Property Prediction: A Teacher-Student Strategy](https://mlsanity.com/paper/2309.04482/) (2023)
- [Efficient Quantization-Aware Distillation with Cross-Modal Alignment for Edge Vision-Language Models](https://mlsanity.com/paper/2609.16689/) (2026)

## Abstract (arXiv)

Model compression and knowledge distillation have been successfully applied for cross-architecture and cross-domain transfer learning. However, a key requirement is that training examples are in correspondence across the domains. We show that in many scenarios of practical importance such aligned data can be synthetically generated using computer graphics pipelines allowing domain adaptation through distillation. We apply this technique to learn models for recognizing low-resolution images using labeled high-resolution images, non-localized objects using labeled localized objects, line-drawings using labeled color images, etc. Experiments on various fine-grained recognition datasets demonstrate that the technique improves recognition performance on the low-quality data and beats strong baselines for domain adaptation. Finally, we present insights into workings of the technique through visualizations and relating it to existing literature.