mlsanityData to 14 Oct 2026

Contrastive Feature Loss for Image Prediction

arXiv 2111.06934 ↗First posted 12 Nov 2021cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result. Yet, this basic functionality remains an open problem. A popular line of approaches uses the L1 (mean absolute error) loss, either in the pixel or the feature space of pretrained deep networks. However, we observe that these losses tend to produce overly blurry and grey images, and other techniques such as GANs need to be employed to fight these artifacts. In this work, we introduce an information theory based approach to measuring similarity between two images. We argue that a good reconstruction should have high mutual information with the ground truth. This view enables learning a lightweight critic to "calibrate" a feature space in a contrastive manner, such that reconstructions of corresponding spatial patches are brought together, while other patches are repulsed. We show that our formulation immediately boosts the perceptual realism of output images when used as a drop-in replacement for the L1 loss, with or without an additional GAN loss.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 38 of its 52 references that are in the map. First posted on arXiv 12 Nov 2021.

Builds on

38 of 52 references.

References · 38

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  10. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  11. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  12. 2019Contrastive Multiview Codingcited by 1,301
26 more
  1. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  2. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  4. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  5. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
  6. 2018Self-Attention Generative Adversarial Networkscited by 809
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  8. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  9. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  10. 2017The Perception-Distortion Tradeoffcited by 587
  11. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
  12. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  13. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  14. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  15. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 195
  16. 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 178
  17. 2016Texture Networks: Feed-forward Synthesis of Textures and Stylized Imagescited by 174
  18. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  19. 2017Playing for Benchmarkscited by 169
  20. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  21. 2019DepthTransfer: Depth Extraction from Video Using Non-parametric Samplingcited by 60
  22. 2017Perceptual Adversarial Networks for Image-to-Image Transformationcited by 42
  23. 2020Semantic Pyramid for Image Generationcited by 34
  24. 2018Image Manipulation with Perceptual Discriminatorscited by 8
  25. 2018Deformation Aware Image Compressioncited by 4
  26. 2020Progressively Unfreezing Perceptual GANcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 1; 2024: 1; 2025: 2; 2026: 1.

12 papers.

  1. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  2. 2023Adaptive Supervised PatchNCE Loss for Learning H&E-to-IHC Stain Translation with Inconsistent Groundtruth Image Pairscited by 38
  3. 2022Modulated Contrast for Versatile Image Synthesiscited by 23
  4. 2022Towards Counterfactual Image Manipulation via CLIPcited by 19
  5. 2022VPTR: Efficient Transformers for Video Predictioncited by 17
  6. 2022Video Prediction by Efficient Transformerscited by 12
  7. 2022Criteria Comparative Learning for Real-scene Image Super-Resolutioncited by 8
  8. 2022Leveraging Local Patch Differences in Multi-Object Scenes for Generative Adversarial Attackscited by 5
  9. 2025Counterfactual Visual Explanation via Causally-Guided Adversarial Steeringcited by 1
  10. 2026CONSTANT: Towards High-Quality One-Shot Handwriting Generation with Patch Contrastive Enhancement and Style-Aware Quantizationcited by 1
  11. 2024Hallucination Detection in Virtually-Stained Histology: A Latent Space Baselinecited by 0
  12. 2025Progressive Translation of H&E to IHC with Enhanced Structural Fidelitycited by 0