# Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks

arXiv: https://arxiv.org/abs/2203.01532 · HTML: https://mlsanity.com/paper/2203.01532/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 30 of 33)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [Correlation Congruence for Knowledge Distillation](https://mlsanity.com/paper/1904.01802/) (2019)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [Photorealistic Style Transfer via Wavelet Transforms](https://mlsanity.com/paper/1903.09760/) (2019)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [Knowledge Distillation Meets Self-Supervision](https://mlsanity.com/paper/2006.07114/) (2020)
- [Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1809.05852/) (2018)
- [One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1706.00826/) (2017)
- [AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries](https://mlsanity.com/paper/2011.08435/) (2020)
- [The Spatially-Correlative Loss for Various Image Translation Tasks](https://mlsanity.com/paper/2104.00854/) (2021)
- [Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2108.04547/) (2021)
- [Complementary Relation Contrastive Distillation](https://mlsanity.com/paper/2103.16367/) (2021)
- [Mutual Contrastive Learning for Visual Representation Learning](https://mlsanity.com/paper/2104.12565/) (2021)
- [CO2: Consistent Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2010.02217/) (2020)

## Built on by (most cited first, 30 of 42)

- [Diffusion-based Image Translation using Disentangled Style and Content Representation](https://mlsanity.com/paper/2209.15264/) (2022)
- [Unpaired Image-to-Image Translation via Neural Schr"odinger Bridge](https://mlsanity.com/paper/2305.15086/) (2023)
- [FreeDrag: Feature Dragging for Reliable Point-based Image Editing](https://mlsanity.com/paper/2307.04684/) (2023)
- [StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translation](https://mlsanity.com/paper/2403.20142/) (2024)
- [Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation](https://mlsanity.com/paper/2308.12968/) (2023)
- [GP-UNIT: Generative Prior for Versatile Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2306.04636/) (2023)
- [Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications](https://mlsanity.com/paper/2309.03048/) (2023)
- [Makeup-Guided Facial Privacy Protection via Untrained Neural Network Priors](https://mlsanity.com/paper/2408.12387/) (2024)
- [Exploring Generative AI for Sim2Real in Driving Data Synthesis](https://mlsanity.com/paper/2404.09111/) (2024)
- [One-for-All: Towards Universal Domain Translation with a Single StyleGAN](https://mlsanity.com/paper/2310.14222/) (2023)
- [Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization](https://mlsanity.com/paper/2407.04245/) (2024)
- [ROMA: Cross-Domain Region Similarity Matching for Unpaired Nighttime Infrared to Daytime Visible Video Translation](https://mlsanity.com/paper/2204.12367/) (2022)
- [Towards Identifiable Unsupervised Domain Translation: A Diversified Distribution Matching Approach](https://mlsanity.com/paper/2401.09671/) (2024)
- [Discriminator-Cooperated Feature Map Distillation for GAN Compression](https://mlsanity.com/paper/2212.14169/) (2022)
- [Towards Pragmatic Semantic Image Synthesis for Urban Scenes](https://mlsanity.com/paper/2305.09726/) (2023)
- [CryoGEM: Physics-Informed Generative Cryo-Electron Microscopy](https://mlsanity.com/paper/2312.02235/) (2023)
- [Patch-wise Graph Contrastive Learning for Image Translation](https://mlsanity.com/paper/2312.08223/) (2023)
- [Training A Small Emotional Vision Language Model for Visual Art Comprehension](https://mlsanity.com/paper/2403.11150/) (2024)
- [Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance](https://mlsanity.com/paper/2306.04396/) (2023)
- [Diff-Oracle: Deciphering Oracle Bone Scripts with Controllable Diffusion Model](https://mlsanity.com/paper/2312.13631/) (2023)
- [Unsupervised Variational Translator for Bridging Image Restoration and High-Level Vision Tasks](https://mlsanity.com/paper/2408.08149/) (2024)
- [USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Staining](https://mlsanity.com/paper/2507.05843/) (2025)
- [Unsupervised Synthetic Image Refinement via Contrastive Learning and Consistent Semantic-Structural Constraints](https://mlsanity.com/paper/2304.12591/) (2023)
- [Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects](https://mlsanity.com/paper/2410.01816/) (2024)
- [CSHNet: A Novel Information Asymmetric Image Translation Method](https://mlsanity.com/paper/2501.10197/) (2025)
- [Exploring Content Relationships for Distilling Efficient GANs](https://mlsanity.com/paper/2212.11091/) (2022)
- [DSI2I: Dense Style for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2212.13253/) (2022)
- [Cross-modal Face- and Voice-style Transfer](https://mlsanity.com/paper/2302.13838/) (2023)
- [An Analysis for Image-to-Image Translation and Style Transfer](https://mlsanity.com/paper/2408.06000/) (2024)
- [Improving Object Detection via Local-global Contrastive Learning](https://mlsanity.com/paper/2410.05058/) (2024)

## Abstract (arXiv)

Recently, contrastive learning-based image translation methods have been proposed, which contrasts different spatial locations to enhance the spatial correspondence. However, the methods often ignore the diverse semantic relation within the images. To address this, here we propose a novel semantic relation consistency (SRC) regularization along with the decoupled contrastive learning, which utilize the diverse semantics by focusing on the heterogeneous semantics between the image patches of a single image. To further improve the performance, we present a hard negative mining by exploiting the semantic relation. We verified our method for three tasks: single-modal and multi-modal image translations, and GAN compression task for image translation. Experimental results confirmed the state-of-art performance of our method in all the three tasks.