# Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

arXiv: https://arxiv.org/abs/2108.04547 · HTML: https://mlsanity.com/paper/2108.04547/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 61 references that are in the map.
- First posted on arXiv 10 Aug 2021.

## Builds on (most cited first, 30 of 50)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)

## Built on by (most cited first, 30 of 45)

- [Unpaired Image-to-Image Translation via Neural Schr"odinger Bridge](https://mlsanity.com/paper/2305.15086/) (2023)
- [Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo](https://mlsanity.com/paper/2203.17248/) (2022)
- [Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks](https://mlsanity.com/paper/2203.01532/) (2022)
- [Modulated Contrast for Versatile Image Synthesis](https://mlsanity.com/paper/2203.09333/) (2022)
- [Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning](https://mlsanity.com/paper/2203.07677/) (2022)
- [StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translation](https://mlsanity.com/paper/2403.20142/) (2024)
- [Synthetic optical coherence tomography angiographs for detailed retinal vessel segmentation without human annotations](https://mlsanity.com/paper/2306.10941/) (2023)
- [ITTR: Unpaired Image-to-Image Translation with Transformers](https://mlsanity.com/paper/2203.16015/) (2022)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Spectral Normalization and Dual Contrastive Regularization for Image-to-Image Translation](https://mlsanity.com/paper/2304.11319/) (2023)
- [Hierarchical Skeleton Meta-Prototype Contrastive Learning with Hard Skeleton Mining for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2307.12917/) (2023)
- [Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications](https://mlsanity.com/paper/2309.03048/) (2023)
- [Maximum Spatial Perturbation Consistency for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2203.12707/) (2022)
- [Rethinking the Paradigm of Content Constraints in Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2211.10867/) (2022)
- [Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2204.11018/) (2022)
- [DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning](https://mlsanity.com/paper/2309.11782/) (2023)
- [Multi-cropping Contrastive Learning and Domain Consistency for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2304.12235/) (2023)
- [TPSeNCE: Towards Artifact-Free Realistic Rain Generation for Deraining and Object Detection in Rain](https://mlsanity.com/paper/2311.00660/) (2023)
- [Rotation-Adaptive Point Cloud Domain Generalization via Intricate Orientation Learning](https://mlsanity.com/paper/2502.02247/) (2025)
- [Energy-Guided Diffusion Model for CBCT-to-CT Synthesis](https://mlsanity.com/paper/2308.03354/) (2023)
- [Probabilistic Self-supervised Learning via Scoring Rules Minimization](https://mlsanity.com/paper/2309.02048/) (2023)
- [CryoGEM: Physics-Informed Generative Cryo-Electron Microscopy](https://mlsanity.com/paper/2312.02235/) (2023)
- [Patch-wise Graph Contrastive Learning for Image Translation](https://mlsanity.com/paper/2312.08223/) (2023)
- [SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for Recommendation](https://mlsanity.com/paper/2405.00287/) (2024)
- [USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Staining](https://mlsanity.com/paper/2507.05843/) (2025)
- [Patch-wise Deep Metric Learning for Unsupervised Low-Dose CT Denoising](https://mlsanity.com/paper/2207.02377/) (2022)
- [Unsupervised Synthetic Image Refinement via Contrastive Learning and Consistent Semantic-Structural Constraints](https://mlsanity.com/paper/2304.12591/) (2023)
- [AdaptSSR: Pre-training User Model with Augmentation-Adaptive Self-Supervised Ranking](https://mlsanity.com/paper/2310.09706/) (2023)
- [Contrastive Learning and Cycle Consistency-based Transductive Transfer Learning for Target Annotation](https://mlsanity.com/paper/2401.12340/) (2024)
- [CAS-GAN for Contrast-free Angiography Synthesis](https://mlsanity.com/paper/2410.08490/) (2024)

## Abstract (arXiv)

Contrastive learning shows great potential in unpaired image-to-image translation, but sometimes the translated results are in poor quality and the contents are not preserved consistently. In this paper, we uncover that the negative examples play a critical role in the performance of contrastive learning for image translation. The negative examples in previous methods are randomly sampled from the patches of different positions in the source image, which are not effective to push the positive examples close to the query examples. To address this issue, we present instance-wise hard Negative Example Generation for Contrastive learning in Unpaired image-to-image Translation (NEGCUT). Specifically, we train a generator to produce negative examples online. The generator is novel from two perspectives: 1) it is instance-wise which means that the generated examples are based on the input image, and 2) it can generate hard negative examples since it is trained with an adversarial loss. With the generator, the performance of unpaired image-to-image translation is significantly improved. Experiments on three benchmark datasets demonstrate that the proposed NEGCUT framework achieves state-of-the-art performance compared to previous methods.