mlsanityData to 14 Oct 2026

Transferring Knowledge with Attention Distillation for Multi-Domain Image-to-Image Translation

arXiv 2108.07466 ↗First posted 17 Aug 2021cs.CV eess.IVcited by 0

Not on the map. Why?

Abstract (arXiv)

Gradient-based attention modeling has been used widely as a way to visualize and understand convolutional neural networks. However, exploiting these visual explanations during the training of generative adversarial networks (GANs) is an unexplored area in computer vision research. Indeed, we argue that this kind of information can be used to influence GANs training in a positive way. For this reason, in this paper, it is shown how gradient based attentions can be used as knowledge to be conveyed in a teacher-student paradigm for multi-domain image-to-image translation tasks in order to improve the results of the student architecture. Further, it is demonstrated how "pseudo"-attentions can also be employed during training when teacher and student networks are trained on different domains which share some similarities. The approach is validated on multi-domain facial attributes transfer and human expression synthesis showing both qualitative and quantitative results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 17 Aug 2021.

Builds on

30 of 39 references.

References · 30

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  5. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  7. 2014Conditional Generative Adversarial Netscited by 2,162
  8. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  9. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  10. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
  11. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
  12. 2019Relational Knowledge Distillationcited by 889
18 more
  1. 2018Self-Attention Generative Adversarial Networkscited by 809
  2. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  3. 2016Generative Adversarial Text to Image Synthesiscited by 735
  4. 2017Unsupervised Image-to-Image Translation Networkscited by 603
  5. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  6. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  7. 2018Model compression via distillation and quantizationcited by 289
  8. 2018Learning without Memorizingcited by 250
  9. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  10. 2018Tell Me Where to Look: Guided Attention Inference Networkcited by 169
  11. 2019Towards Visually Explaining Variational Autoencoderscited by 113
  12. 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 109
  13. 2016Scribbler: Controlling Deep Image Synthesis with Sketch and Colorcited by 78
  14. 2019RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributescited by 56
  15. 2017TextureGAN: Controlling Deep Image Synthesis with Texture Patchescited by 47
  16. 2019Learning Fixed Points in Generative Adversarial Networks: From Image-to-Image Translation to Disease Detection and Localizationcited by 38
  17. 2019Compressing GANs using Knowledge Distillationcited by 31
  18. 2019MetalGAN: Multi-Domain Label-Less Image Synthesis Using cGANs and Meta-Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Region-aware Knowledge Distillation for Efficient Image-to-Image Translation
  2. 2023Few-shot Face Image Translation via GAN Prior Distillation
  3. 2021Semantic Relation Preserving Knowledge Distillation for Image-to-Image Translation
  4. 2025Attention Distillation: A Unified Approach to Visual Characteristics Transfer
  5. 2023A Simple and Effective Baseline for Attentional Generative Adversarial Networks
  6. 2022Teaching Where to Look: Attention Similarity Knowledge Distillation for Low Resolution Face Recognition
  7. 2025Learning Object Focused Attention
  8. 2026Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation