mlsanityData to 14 Oct 2026

StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models

arXiv 2308.07863 ↗First posted 15 Aug 2023cs.CVcited by 155

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Content and style (C-S) disentanglement is a fundamental problem and critical challenge of style transfer. Existing approaches based on explicit definitions (e.g., Gram matrix) or implicit learning (e.g., GANs) are neither interpretable nor easy to control, resulting in entangled representations and less satisfying results. In this paper, we propose a new C-S disentangled framework for style transfer without using previous assumptions. The key insight is to explicitly extract the content information and implicitly learn the complementary style information, yielding interpretable and controllable C-S disentanglement and style transfer. A simple yet effective CLIP-based style disentanglement loss coordinated with a style reconstruction prior is introduced to disentangle C-S in the CLIP image space. By further leveraging the powerful style removal and generative ability of diffusion models, our framework achieves superior results than state of the art and flexible C-S disentanglement and trade-off control. Our work provides new insights into the C-S disentanglement in style transfer and demonstrates the potential of diffusion models for learning well-disentangled C-S characteristics.

Key facts. Cited by 155 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 80 of its 106 references that are in the map. First posted on arXiv 15 Aug 2023.

Builds on

80 of 106 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  5. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  6. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 74

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  9. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  10. 2020Denoising Diffusion Implicit Modelscited by 9,781
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  12. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
38 more
  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  9. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  10. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  11. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  12. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  13. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  14. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  15. 2018Disentangling by Factorisingcited by 616
  16. 2015Holistically-Nested Edge Detectioncited by 592
  17. 2017Universal Style Transfer via Feature Transformscited by 354
  18. 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
  19. 2018Diverse Image-to-Image Translation via Disentangled Representationscited by 296
  20. 2018Towards a Definition of Disentangled Representationscited by 296
  21. 2020A Survey on Neural Network Interpretabilitycited by 291
  22. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  23. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  24. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  25. 2017Neural Style Transfer: A Reviewcited by 199
  26. 2018Arbitrary Style Transfer with Style-Attentional Networkscited by 198
  27. 2021StyTr2: Image Style Transfer with Transformerscited by 187
  28. 2021AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfercited by 178
  29. 2017Deep Photo Style Transfercited by 168
  30. 2022Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalizationcited by 122
  31. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  32. 2018Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decorationcited by 115
  33. 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
  34. 2022Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learningcited by 93
  35. 2017StyleBank: An Explicit Representation for Neural Image Style Transfercited by 89
  36. 2019Dynamic Instance Normalization for Arbitrary Style Transfercited by 83
  37. 2018Deep Exemplar-based Colorizationcited by 82
  38. 2018Arbitrary Style Transfer with Deep Feature Reshufflecited by 75

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 63; 2025: 61; 2026: 23.

155 papers.

Sort:

By area:

  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
  3. 2024StyleShot: A Snapshot on Any Stylecited by 62
  4. 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
  5. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  6. 2024Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editingcited by 39
  7. 2024EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Modelscited by 37
  8. 2024AeroGen: Enhancing Remote Sensing Object Detection with Diffusion-Driven Data Generationcited by 35
  9. 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?cited by 33
  10. 2025Attention Distillation: A Unified Approach to Visual Characteristics Transfercited by 33
  11. 2025OmniStyle: Filtering High Quality Style Transfer Data at Scalecited by 30
  12. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
38 more
  1. 2024BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Modelscited by 29
  2. 2024Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimizationcited by 26
  3. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  4. 2024FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Modelscited by 22
  5. 2024Source Prompt Disentangled Inversion for Boosting Image Editability with Diffusion Modelscited by 22
  6. 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
  7. 2025Multi-turn Consistent Image Editingcited by 18
  8. 2023InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Advisercited by 17
  9. 2024PROUD: PaRetO-gUided Diffusion Model for Multi-objective Generationcited by 16
  10. 2025DreamPoster: A Unified Framework for Image-Conditioned Generative Poster Designcited by 16
  11. 2024Generating Compositional Scenes via Text-to-image RGBA Instance Generationcited by 14
  12. 2025Diffusion Models Through a Global Lens: Are They Culturally Inclusive?cited by 14
  13. 2025Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Modelscited by 14
  14. 2025ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translationcited by 14
  15. 2024ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generationcited by 13
  16. 2024Multi-Modal and Multi-Attribute Generation of Single Cells with CFGencited by 12
  17. 2024Novel Object Synthesis via Adaptive Text-Image Harmonycited by 12
  18. 2025U-StyDiT: Ultra-high Quality Artistic Style Transfer Using Diffusion Transformerscited by 12
  19. 2025Cross-Frame Representation Alignment for Fine-Tuning Video Diffusion Modelscited by 12
  20. 2025Scene4U: Hierarchical Layered 3D Scene Reconstruction from Single Panoramic Image for Your Immerse Explorationcited by 11
  21. 2023Portrait Diffusion: Training-free Face Stylization with Chain-of-Paintingcited by 10
  22. 2024MuseumMaker: Continual Style Customization without Catastrophic Forgettingcited by 10
  23. 2024DiffVax: Optimization-Free Image Immunization Against Diffusion-Based Editingcited by 10
  24. 2024Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalizationcited by 9
  25. 2024Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedbackcited by 9
  26. 2025AttenST: A Training-Free Attention-Driven Style Transfer Framework with Pre-Trained Diffusion Modelscited by 9
  27. 2024EasyInv: Toward Fast and Better DDIM Inversioncited by 8
  28. 2024SEED4D: A Synthetic Ego--Exo Dynamic 4D Data Generator, Driving Dataset and Benchmarkcited by 8
  29. 2025FreeBlend: Advancing Concept Blending with Staged Feedback-Driven Interpolation Diffusioncited by 8
  30. 2025Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylizationcited by 8
  31. 2025Free-Lunch Color-Texture Disentanglement for Stylized Image Generationcited by 8
  32. 2023Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generationcited by 7
  33. 2024U-VAP: User-specified Visual Appearance Personalization via Decoupled Self Augmentationcited by 7
  34. 2024Diffusion-based Human Motion Style Transfer with Semantic Guidancecited by 7
  35. 2024U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AIcited by 7
  36. 2025GAITGen: Disentangled Motion-Pathology Impaired Gait Generative Model -- Bringing Motion Generation to the Clinical Domaincited by 7
  37. 2024DiffArtist: Towards Structure and Appearance Controllable Image Stylizationcited by 6
  38. 2024FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Processcited by 6

50 most cited of 155 All 155 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Diffusion-based Image Translation using Disentangled Style and Content Representation
  2. 2026TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer
  3. 2026UniCSG: Unified High-Fidelity Content-Constrained Style-Driven Generation via Staged Semantic and Frequency Disentanglement
  4. 2026Scheduled Style Injection: Expanding the Style-Content Pareto Frontier in Training-Free Diffusion-based Style Transfer
  5. 2024Z-STAR+: A Zero-shot Style Transfer Method via Adjusting Style Distribution
  6. 2023Z^*: Zero-shot Style Transfer via Attention Rearrangement
  7. 2025SigStyle: Signature Style Transfer via Personalized Text-to-Image Models
  8. 2026Abstract-LoRA: Unlocking Single-Image Style Transfer through Targeted U-Net Block Training