mlsanityData to 7 Oct 2026

DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations

arXiv 2403.06951 ↗First posted 11 Mar 2024cs.CVcited by 68

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

The diffusion-based text-to-image model harbors immense potential in transferring reference style. However, current encoder-based approaches significantly impair the text controllability of text-to-image models while transferring styles. In this paper, we introduce DEADiff to address this issue using the following two strategies: 1) a mechanism to decouple the style and semantics of reference images. The decoupled feature representations are first extracted by Q-Formers which are instructed by different text descriptions. Then they are injected into mutually exclusive subsets of cross-attention layers for better disentanglement. 2) A non-reconstructive learning method. The Q-Formers are trained using paired images rather than the identical target, in which the reference image and the ground-truth image are with the same style or semantics. We show that DEADiff attains the best visual stylization results and optimal balance between the text controllability inherent in the text-to-image model and style similarity to the reference image, as demonstrated both quantitatively and qualitatively. Our project page is https://tianhao-qi.github.io/DEADiff/.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 31 of its 38 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

31 of 38 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  2. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  6. 2022Inversion-Based Style Transfer with Diffusion Models23 Nov
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
10 more
  1. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  2. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  3. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  4. 2018Arbitrary Style Transfer with Style-Attentional Networkscited by 197
  5. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 197
  6. 2021StyTr2: Image Style Transfer with Transformerscited by 186
  7. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 167
  8. 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
  9. 2022Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learningcited by 93
  10. 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 32; 2026: 19.

68 papers.

Sort:

By area:

  1. 2024InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generationcited by 166
  2. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  3. 2024CSGO: Content-Style Composition in Text-to-Image Generationcited by 76
  4. 2024StyleShot: A Snapshot on Any Stylecited by 60
  5. 2024InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generationcited by 51
  6. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  7. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Modelscited by 27
  8. 2025USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learningcited by 26
  9. 2024FreeTuner: Any Subject in Any Style with Training-free Diffusioncited by 24
  10. 2025StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfercited by 23
  11. 2025Nested Attention: Semantic-aware Attention Values for Concept Personalizationcited by 21
  12. 2024Diffusion-Based Visual Art Creation: A Survey and New Perspectivescited by 17
38 more
  1. 2024StyleTex: Style Image-Guided Texture Generation for 3D Modelscited by 14
  2. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  3. 2025Context-guided Responsible Data Augmentation with Diffusion Modelscited by 9
  4. 2025Free-Lunch Color-Texture Disentanglement for Stylized Image Generationcited by 8
  5. 2025DRC: Enhancing Personalized Image Generation via Disentangled Representation Compositioncited by 8
  6. 2025Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioningcited by 7
  7. 2025Style Transfer with Diffusion Models for Synthetic-to-Real Domain Adaptationcited by 6
  8. 2025Training-Free Multi-Style Fusion Through Reference-Based Adaptive Modulationcited by 6
  9. 2025SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Modelscited by 5
  10. 2025ArtCrafter: Text-Image Aligning Style Transfer via Embedding Reframingcited by 4
  11. 2025A Comprehensive Survey on Visual Concept Mining in Text-to-image Diffusion Modelscited by 4
  12. 2026MegaStyle: Constructing Diverse and Scalable Style Dataset via Consistent Text-to-Image Style Mappingcited by 4
  13. 2024FiVA: Fine-grained Visual Attribute Dataset for Text-to-Image Diffusion Modelscited by 3
  14. 2024OmniPrism: Learning Disentangled Visual Concept for Image Generationcited by 3
  15. 2025SGD-Mix: Enhancing Domain-Specific Image Classification with Label-Preserving Data Augmentationcited by 3
  16. 2026DreamStyle: A Unified Framework for Video Stylizationcited by 3
  17. 2026CleanStyle: Plug-and-Play Style Conditioning Purification for Text-to-Image Stylizationcited by 3
  18. 2025Efficient Temporal Consistency in Diffusion-Based Video Editing with Adaptor Modules: A Theoretical Frameworkcited by 2
  19. 2025Training-free Stylized Text-to-Image Generation with Fast Inferencecited by 2
  20. 2025TransLight: Image-Guided Customized Lighting Control with Generative Decouplingcited by 2
  21. 2025GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfercited by 2
  22. 2025Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relightingcited by 2
  23. 2026Semi-supervised Latent Disentangled Diffusion Model for Textile Pattern Generationcited by 2
  24. 2026Scale-Separated Conditioning for Style-Encoder-Free Diffusion Stylizationcited by 2
  25. 2024CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editingcited by 1
  26. 2024WikiStyle+: A Multimodal Approach to Content-Style Representation Disentanglement for Artistic Image Stylizationcited by 1
  27. 2025Gungnir: Exploiting Stylistic Features in Images for Backdoor Attacks on Diffusion Modelscited by 1
  28. 2025Semantix: An Energy Guided Sampler for Semantic Style Transfercited by 1
  29. 2025SPF-Portrait: Towards Pure Text-to-Portrait Customization with Semantic Pollution-Free Fine-Tuningcited by 1
  30. 2025OmniStyle2: Learning to Stylize by Learning to Destylizecited by 1
  31. 2026Compressing Image Style Training into a Single Model Forwardcited by 1
  32. 2024StyleBrush: Style Extraction and Transfer from a Single Imagecited by 0
  33. 2024Improving Fine-Grained Control via Aggregation of Multiple Diffusion Modelscited by 0
  34. 2024Opt-In Art: Learning Art Styles Only from Few Examplescited by 0
  35. 2025ICAS: IP Adapter and ControlNet-based Attention Structure for Multi-Subject Style Transfer Optimizationcited by 0
  36. 2025Controllable Coupled Image Generation via Diffusion Modelscited by 0
  37. 2025MFGDiffusion: Mask-Guided Smoke Synthesis for Enhanced Forest Fire Detectioncited by 0
  38. 2025One-shot Embroidery Customization via Contrastive LoRA Modulationcited by 0

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025StyDeco: Unsupervised Style Transfer with Distilling Priors and Semantic Decoupling
  2. 2022Diffusion-based Image Translation using Disentangled Style and Content Representation
  3. 2025Leveraging Diffusion Models for Stylization using Multiple Style Images
  4. 2025Balanced Image Stylization with Style Matching Score
  5. 2026CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer
  6. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models
  7. 2024ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model
  8. 2024DiffuseST: Unleashing the Capability of the Diffusion Model for Style Transfer