mlsanityData to 7 Oct 2026

Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer

arXiv 2303.08622 ↗First posted 15 Mar 2023cs.CV cs.AI cs.LGcited by 59

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have shown great promise in text-guided image style transfer, but there is a trade-off between style transformation and content preservation due to their stochastic nature. Existing methods require computationally expensive fine-tuning of diffusion models or additional neural network. To address this, here we propose a zero-shot contrastive loss for diffusion models that doesn't require additional fine-tuning or auxiliary networks. By leveraging patch-wise contrastive loss between generated samples and original image embeddings in the pre-trained diffusion model, our method can generate images with the same semantic content as the source image in a zero-shot manner. Our approach outperforms existing methods while preserving content and requiring no additional training, not only for image style transfer but also for image-to-image translation and manipulation. Our experimental results validate the effectiveness of our proposed method.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 36 of its 44 references that are in the map. First posted on arXiv 15 Mar 2023.

Builds on

36 of 44 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  2. 2022Diffusion-based Image Translation using Disentangled Style and Content Representation30 Sep
  3. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translation16 Mar
  4. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance10 Dec
  5. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  6. 2021Palette: Image-to-Image Diffusion Models10 Nov
  7. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
17 more
  1. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  5. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  6. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  7. 2023Zero-shot Image-to-Image Translationcited by 503
  8. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  9. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  10. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  11. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  12. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  13. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equationscited by 172
  14. 2019Photorealistic Style Transfer via Wavelet Transformscited by 166
  15. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  16. 2021Language-Driven Image Style Transfercited by 36
  17. 2016SANet: Structure-Aware Network for Visual Trackingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 19; 2025: 21; 2026: 3.

59 papers.

Sort:

By area:

  1. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 194
  2. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  3. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75
  4. 2024StyleShot: A Snapshot on Any Stylecited by 60
  5. 2023Contrastive Denoising Score for Text-guided Latent Diffusion Image Editingcited by 49
  6. 2023Towards Accurate Post-training Quantization for Diffusion Modelscited by 46
  7. 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Modelscited by 46
  8. 2024Language-driven Grasp Detectioncited by 26
  9. 2024FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Modelscited by 21
  10. 2023Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaboratecited by 17
  11. 2023InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Advisercited by 17
  12. 2023Z^*: Zero-shot Style Transfer via Attention Rearrangementcited by 17
38 more
  1. 2023Free-Editor: Zero-shot Text-driven 3D Scene Editingcited by 14
  2. 2024GDA: Generalized Diffusion for Robust Test-time Adaptationcited by 14
  3. 2023Using Diffusion Models to Generate Synthetic Labelled Data for Medical Image Segmentationcited by 10
  4. 2023Portrait Diffusion: Training-free Face Stylization with Chain-of-Paintingcited by 10
  5. 2024Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalizationcited by 9
  6. 2025QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generationcited by 9
  7. 2025AttenST: A Training-Free Attention-Driven Style Transfer Framework with Pre-Trained Diffusion Modelscited by 8
  8. 2023Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generationcited by 7
  9. 2024High-Resolution Frame Interpolation with Patch-based Cascaded Diffusioncited by 7
  10. 2025Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priorscited by 6
  11. 2024StyleMamba : State Space Model for Efficient Text-driven Image Style Transfercited by 5
  12. 2024Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Trainingcited by 5
  13. 2024DiffuseST: Unleashing the Capability of the Diffusion Model for Style Transfercited by 5
  14. 2025Balanced Image Stylization with Style Matching Scorecited by 5
  15. 2023Dequantization and Color Transfer with Diffusion Modelscited by 4
  16. 2025SCSA: A Plug-and-Play Semantic Continuous-Sparse Attention for Arbitrary Semantic Style Transfercited by 4
  17. 2025Attention in Diffusion Model: A Surveycited by 4
  18. 2025A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Modelcited by 4
  19. 2026MegaStyle: Constructing Diverse and Scalable Style Dataset via Consistent Text-to-Image Style Mappingcited by 4
  20. 2024Font Style Interpolation with Diffusion Modelscited by 3
  21. 2025Contrastive Learning Guided Latent Diffusion Model for Image-to-Image Translationcited by 3
  22. 2025SPAST: Arbitrary Style Transfer with Style Priors via Pre-trained Large-scale Modelcited by 3
  23. 2023Improving Diffusion-based Image Translation using Asymmetric Gradient Guidancecited by 2
  24. 2024StyleCity: Large-Scale 3D Urban Scenes Stylizationcited by 2
  25. 2024ZePo: Zero-Shot Portrait Stylization with Faster Samplingcited by 2
  26. 2024Rethinking Clothes Changing Person ReID: Conflicts, Synthesis, and Optimizationcited by 1
  27. 2025UStyle: Waterbody Style Transfer of Underwater Scenes by Depth-Guided Feature Synthesiscited by 1
  28. 2025Semantix: An Energy Guided Sampler for Semantic Style Transfercited by 1
  29. 2025Training-Free Diffusion Framework for Stylized Image Generation with Identity Preservationcited by 1
  30. 2025MTADiffusion: Mask Text Alignment Diffusion Model for Object Inpaintingcited by 1
  31. 2025DCDB: Dynamic Conditional Dual Diffusion Bridge for Ill-posed Multi-Taskscited by 1
  32. 2025SCAdapter: Content-Style Disentanglement for Diffusion Style Transfercited by 1
  33. 2023Diffusion Cocktail: Mixing Domain-Specific Diffusion Models for Diversified Image Generationscited by 0
  34. 2023DXAI: Explaining Classification by Image Decompositioncited by 0
  35. 2024Cross-Domain Image Conversion by CycleDMcited by 0
  36. 2024LEAST: "Local" text-conditioned image style transfercited by 0
  37. 2024FAGStyle: Feature Augmentation on Geodesic Surface for Zero-shot Text-guided Diffusion Image Style Transfercited by 0
  38. 2024Bridging Text and Image for Artist Style Transfer via Contrastive Learningcited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Text-to-Image Diffusion Models are Zero-Shot Classifiers
  2. 2025Structure-Preserving Zero-Shot Image Editing via Stage-Wise Latent Injection in Diffusion Models
  3. 2024Z-STAR+: A Zero-shot Style Transfer Method via Adjusting Style Distribution
  4. 2023Filter-Guided Diffusion for Controllable Image Generation
  5. 2025Less is More: Masking Elements in Image Condition Features Avoids Content Leakages in Style Transfer Diffusion Models
  6. 2023ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priors
  7. 2023DIFF-NST: Diffusion Interleaving For deFormable Neural Style Transfer
  8. 2023ReGeneration Learning of Diffusion Models with Rich Prompts for Zero-Shot Image Translation