mlsanityData to 14 Oct 2026

The Spatially-Correlative Loss for Various Image Translation Tasks

arXiv 2104.00854 ↗First posted 2 Apr 2021cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency while supporting large appearance changes during unpaired image-to-image (I2I) translation. Previous methods attempt this by using pixel-level cycle-consistency or feature-level matching losses, but the domain-specific nature of these losses hinder translation across large domain gaps. To address this, we exploit the spatial patterns of self-similarity as a means of defining scene structure. Our spatially-correlative loss is geared towards only capturing spatial relationships within an image rather than domain appearance. We also introduce a new self-supervised learning method to explicitly learn spatially-correlative maps for each specific translation task. We show distinct improvement over baseline models in all three modes of unpaired I2I translation: single-modal, multi-modal, and even single-image translation. This new loss can easily be integrated into existing network architectures and thus allows wide applicability.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 48 of its 59 references that are in the map. First posted on arXiv 2 Apr 2021.

Builds on

48 of 59 references.

References · 48

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  10. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  11. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
36 more
  1. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  2. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  3. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  4. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  5. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  6. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  7. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
  8. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  9. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  10. 2018Which Training Methods for GANs do actually Converge?cited by 679
  11. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  12. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 639
  13. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 434
  14. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  15. 2017DualGAN: Unsupervised Dual Learning for Image-to-Image Translationcited by 310
  16. 2017Toward Multimodal Image-to-Image Translationcited by 296
  17. 2017Dilated Residual Networkscited by 294
  18. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 288
  19. 2017Learning to Discover Cross-Domain Relations with Generative Adversarial Networkscited by 278
  20. 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
  21. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  22. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 222
  23. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  24. 2017Deep Photo Style Transfercited by 168
  25. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 168
  26. 2019Photorealistic Style Transfer via Wavelet Transformscited by 166
  27. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  28. 2018Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mappingcited by 113
  29. 2017One-Sided Unsupervised Domain Mappingcited by 91
  30. 2020TSIT: A Simple and Versatile Framework for Image-to-Image Translationcited by 64
  31. 2018Unsupervised Attention-guided Image to Image Translationcited by 58
  32. 2020Rethinking the Truly Unsupervised Image-to-Image Translationcited by 56
  33. 2019Breaking the cycle -- Colleagues are all you needcited by 40
  34. 2019TraVeLGAN: Image-to-image Translation by Transformation Vector Learningcited by 35
  35. 2018Attention-GAN for Object Transfiguration in Wild Imagescited by 32
  36. 2020Deformable Style Transfercited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 18; 2023: 23; 2024: 9; 2025: 7; 2026: 3.

66 papers.

Sort:

By area:

  1. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equationscited by 172
  2. 2023Unpaired Image-to-Image Translation via Neural Schr"odinger Bridgecited by 96
  3. 2021Relational Embedding for Few-Shot Classificationcited by 74
  4. 2021StyleAlign: Analysis and Applications of Aligned StyleGAN Modelscited by 48
  5. 2022Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Taskscited by 42
  6. 2022QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translationcited by 39
  7. 2023ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bankcited by 36
  8. 2022Look Closer to Supervise Better: One-Shot Font Generation via Component-Based Discriminatorcited by 32
  9. 2023General Image-to-Image Translation with One-Shot Image Guidancecited by 31
  10. 2022InstaFormer: Instance-Aware Image-to-Image Translation with Transformercited by 28
  11. 2022Structure Representation Network and Uncertainty Feedback Learning for Dense Non-Uniform Fog Removalcited by 28
  12. 2022Modulated Contrast for Versatile Image Synthesiscited by 23
38 more
  1. 2022Partially Does It: Towards Scene-Level FG-SBIR with Partial Inputcited by 20
  2. 2022Marginal Contrastive Correspondence for Guided Image Generationcited by 19
  3. 2021Robustness via Uncertainty-aware Cycle Consistencycited by 16
  4. 2024Towards Highly Realistic Artistic Style Transfer via Stable Diffusion with Step-aware and Layer-aware Promptcited by 16
  5. 2022ITTR: Unpaired Image-to-Image Translation with Transformerscited by 15
  6. 2023SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translationcited by 14
  7. 2023Spectral Normalization and Dual Contrastive Regularization for Image-to-Image Translationcited by 11
  8. 2023Probabilistic Precision and Recall Towards Reliable Evaluation of Generative Modelscited by 11
  9. 2023Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applicationscited by 9
  10. 2022Maximum Spatial Perturbation Consistency for Unpaired Image-to-Image Translationcited by 8
  11. 2023UVCGAN v2: An Improved Cycle-Consistent GAN for Unpaired Image-to-Image Translationcited by 8
  12. 2024C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generationcited by 8
  13. 2022Rethinking the Paradigm of Content Constraints in Unpaired Image-to-Image Translationcited by 7
  14. 2025CycleDiff: Cycle Diffusion Models for Unpaired Image-to-image Translationcited by 7
  15. 2021ManiFest: Manifold Deformation for Few-shot Image Translationcited by 6
  16. 2022Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translationcited by 6
  17. 2023ReGeneration Learning of Diffusion Models with Rich Prompts for Zero-Shot Image Translationcited by 6
  18. 2024Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalizationcited by 6
  19. 2022ROMA: Cross-Domain Region Similarity Matching for Unpaired Nighttime Infrared to Daytime Visible Video Translationcited by 5
  20. 2022Ultra-high-resolution unpaired stain transformation via Kernelized Instance Normalizationcited by 5
  21. 2022LANIT: Language-Driven Image-to-Image Translation for Unlabeled Datacited by 5
  22. 2023Multi-cropping Contrastive Learning and Domain Consistency for Unsupervised Image-to-Image Translationcited by 5
  23. 2023Fix the Noise: Disentangling Source Feature for Controllable Domain Translationcited by 4
  24. 2023Towards Pragmatic Semantic Image Synthesis for Urban Scenescited by 4
  25. 2023CryoGEM: Physics-Informed Generative Cryo-Electron Microscopycited by 4
  26. 2023Patch-wise Graph Contrastive Learning for Image Translationcited by 4
  27. 2024Global-Local Similarity for Efficient Fine-Grained Image Recognition with Vision Transformerscited by 4
  28. 2025Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillationcited by 4
  29. 2025USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Stainingcited by 3
  30. 2026StyMam: A Mamba-Based Generator for Artistic Style Transfercited by 3
  31. 2021Content Preserving Image Translation with Texture Co-occurrence and Spatial Self-Similarity for Texture Debiasing and Domain Adaptationcited by 2
  32. 2023Unpaired Translation from Semantic Label Maps to Images by Leveraging Domain-Specific Simulationscited by 2
  33. 2023Unsupervised Synthetic Image Refinement via Contrastive Learning and Consistent Semantic-Structural Constraintscited by 2
  34. 2023Deep Transductive Transfer Learning for Automatic Target Recognitioncited by 2
  35. 2023Unpaired Multi-domain Attribute Translation of 3D Facial Shapes with a Square and Symmetric Geometric Mapcited by 2
  36. 2024Contrastive Learning and Cycle Consistency-based Transductive Transfer Learning for Target Annotationcited by 2
  37. 2025Learning to Match Unpaired Data with Minimum Entropy Couplingcited by 2
  38. 2026Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridgecited by 2

50 most cited of 66 All 66 (JSON)