# Memory-guided Unsupervised Image-to-image Translation

arXiv: https://arxiv.org/abs/2104.05170 · HTML: https://mlsanity.com/paper/2104.05170/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 53 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Image to Image Translation for Domain Adaptation](https://mlsanity.com/paper/1712.00479/) (2017)
- [Unsupervised Cross-Domain Image Generation](https://mlsanity.com/paper/1611.02200/) (2016)

## Built on by (most cited first, 21 of 21)

- [Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation](https://mlsanity.com/paper/2203.06321/) (2022)
- [InstaFormer: Instance-Aware Image-to-Image Translation with Transformer](https://mlsanity.com/paper/2203.16248/) (2022)
- [Robustness via Uncertainty-aware Cycle Consistency](https://mlsanity.com/paper/2110.12467/) (2021)
- [ADAS: A Direct Adaptation Strategy for Multi-Target Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2203.06811/) (2022)
- [Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation](https://mlsanity.com/paper/2309.16127/) (2023)
- [GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data](https://mlsanity.com/paper/2207.13297/) (2022)
- [Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation](https://mlsanity.com/paper/2602.15383/) (2026)
- [LANIT: Language-Driven Image-to-Image Translation for Unlabeled Data](https://mlsanity.com/paper/2208.14889/) (2022)
- [MuTri: Multi-view Tri-alignment for OCT to OCTA 3D Image Translation](https://mlsanity.com/paper/2504.01428/) (2025)
- [SHUNIT: Style Harmonization for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2301.04685/) (2023)
- [3D-Aware Multi-Class Image-to-Image Translation with NeRFs](https://mlsanity.com/paper/2303.15012/) (2023)
- [Masked Discriminators for Content-Consistent Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2309.13188/) (2023)
- [Leveraging Local Domains for Image-to-Image Translation](https://mlsanity.com/paper/2109.04468/) (2021)
- [Memory-Guided Collaborative Attention for Nighttime Thermal Infrared Image Colorization](https://mlsanity.com/paper/2208.02960/) (2022)
- [DSI2I: Dense Style for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2212.13253/) (2022)
- [Panoramic Image-to-Image Translation](https://mlsanity.com/paper/2304.04960/) (2023)
- [Improving Object Detection via Local-global Contrastive Learning](https://mlsanity.com/paper/2410.05058/) (2024)
- [Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution](https://mlsanity.com/paper/2202.02779/) (2022)
- [Disentangled Contrastive Image Translation for Nighttime Surveillance](https://mlsanity.com/paper/2307.05038/) (2023)
- [RainSD: Rain Style Diversification Module for Image Synthesis Enhancement using Feature-Level Style Distribution](https://mlsanity.com/paper/2401.00460/) (2023)
- [SafePaint: Anti-forensic Image Inpainting with Domain Adaptation](https://mlsanity.com/paper/2404.18136/) (2024)

## Abstract (arXiv)

We present a novel unsupervised framework for instance-level image-to-image translation. Although recent advances have been made by incorporating additional object annotations, existing methods often fail to handle images with multiple disparate objects. The main cause is that, during inference, they apply a global style to the whole image and do not consider the large style discrepancy between instance and background, or within instances. To address this problem, we propose a class-aware memory network that explicitly reasons about local style variations. A key-values memory structure, with a set of read/update operations, is introduced to record class-wise style variations and access them without requiring an object detector at the test time. The key stores a domain-agnostic content representation for allocating memory items, while the values encode domain-specific style representations. We also present a feature contrastive loss to boost the discriminative power of memory items. We show that by incorporating our memory, we can transfer class-aware and accurate style representations across domains. Experimental results demonstrate that our model outperforms recent instance-level methods and achieves state-of-the-art performance.