mlsanityData to 7 Oct 2026

Semantic Relation Preserving Knowledge Distillation for Image-to-Image Translation

arXiv 2104.15082 ↗First posted 30 Apr 2021cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Generative adversarial networks (GANs) have shown significant potential in modeling high dimensional distributions of image data, especially on image-to-image translation tasks. However, due to the complexity of these tasks, state-of-the-art models often contain a tremendous amount of parameters, which results in large model size and long inference time. In this work, we propose a novel method to address this problem by applying knowledge distillation together with distillation of a semantic relation preserving matrix. This matrix, derived from the teacher's feature encoding, helps the student model learn better semantic relations. In contrast to existing compression methods designed for classification tasks, our proposed method adapts well to the image-to-image translation task on GANs. Experiments conducted on 5 different datasets and 3 different pairs of teacher and student models provide strong evidence that our methods achieve impressive results both qualitatively and quantitatively.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 28 of its 33 references that are in the map. First posted on arXiv 30 Apr 2021.

Builds on

28 of 33 references.

References · 28

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2018YOLOv3: An Incremental Improvementcited by 2,652
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2019Searching for MobileNetV3cited by 1,984
  12. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
16 more
  1. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  2. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  3. 2019Relational Knowledge Distillationcited by 873
  4. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  5. 2019Similarity-Preserving Knowledge Distillationcited by 507
  6. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  7. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  8. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  9. 2019Correlation Congruence for Knowledge Distillationcited by 280
  10. 2019Structured Knowledge Distillation for Dense Predictioncited by 277
  11. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  12. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 208
  13. 2019DetNAS: Backbone Search for Object Detectioncited by 80
  14. 2017DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfercited by 55
  15. 2019Co-Evolutionary Compression for Unpaired Image Translationcited by 32
  16. 2019Compressing GANs using Knowledge Distillationcited by 31

Built on by

Cited by year

Citing papers per year: 2022: 7; 2023: 2; 2024: 1; 2025: 3.

13 papers.

Sort:

By area:

  1. 2022Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translationcited by 39
  2. 2022PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detectioncited by 33
  3. 2022Region-aware Knowledge Distillation for Efficient Image-to-Image Translationcited by 10
  4. 2022Discriminator-Cooperated Feature Map Distillation for GAN Compressioncited by 4
  5. 2022Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detectioncited by 3
  6. 2024DISC: Latent Diffusion Models with Self-Distillation from Separated Conditions for Prostate Cancer Gradingcited by 3
  7. 2025Knowledge Distillation for Image Restoration : Simultaneous Learning from Degraded and Clean Imagescited by 3
  8. 2025Digital Staining with Knowledge Distillation: A Unified Framework for Unpaired and Paired-But-Misaligned Datacited by 2
  9. 2022Learning to Structure an Image with Few Colors and Beyondcited by 1
  10. 2022Exploring Content Relationships for Distilling Efficient GANscited by 1
  11. 2025Semantic-Supervised Spatial-Temporal Fusion for LiDAR-based 3D Object Detectioncited by 1
  12. 2023Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Studycited by 0
1 more
  1. 2023Noise-Tolerant Few-Shot Unsupervised Adapter for Vision-Language Modelscited by 0