mlsanityData to 14 Oct 2026

Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis

arXiv 2304.10278 ↗First posted 20 Apr 2023cs.CVcited by 18

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

The duality of content and style is inherent to the nature of art. For humans, these two elements are clearly different: content refers to the objects and concepts in the piece of art, and style to the way it is expressed. This duality poses an important challenge for computer vision. The visual appearance of objects and concepts is modulated by the style that may reflect the author's emotions, social trends, artistic movement, etc., and their deep comprehension undoubtfully requires to handle both. A promising step towards a general paradigm for art analysis is to disentangle content and style, whereas relying on human annotations to cull a single aspect of artworks has limitations in learning semantic concepts and the visual appearance of paintings. We thus present GOYA, a method that distills the artistic knowledge captured in a recent generative model to disentangle content and style. Experiments show that synthetically generated images sufficiently serve as a proxy of the real distribution of artworks, allowing GOYA to separately represent the two elements of art while keeping more information than existing methods.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 56 of its 90 references that are in the map. First posted on arXiv 20 Apr 2023.

Builds on

56 of 90 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models16 Dec
  2. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clones16 Dec
  3. 2022Diffusion-based Image Translation using Disentangled Style and Content Representation30 Sep
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  6. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  12. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
38 more
  1. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  2. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  3. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  4. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  5. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  6. 2017Large Batch Training of Convolutional Networkscited by 535
  7. 2015A Neural Algorithm of Artistic Stylecited by 483
  8. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  9. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  10. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  11. 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 182
  12. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 162
  13. 2017Unsupervised Learning of Disentangled Representations from Videocited by 139
  14. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 128
  15. 2018How to Read Paintings: Semantic Art Understanding with Multi-Modal Retrievalcited by 74
  16. 2020Arbitrary Style Transfer via Multi-Adaptation Networkcited by 70
  17. 2013Recognizing Image Stylecited by 64
  18. 2018Image-to-image translation for cross-domain disentanglementcited by 61
  19. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  20. 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
  21. 2017BAM! The Behance Artistic Media Dataset for Recognition Beyond Photographycited by 43
  22. 2022The Met Dataset: Instance-level Recognition for Artworkscited by 40
  23. 2019Multi-mapping Image-to-Image Translation via Learning Disentanglementcited by 35
  24. 2021ALADIN: All Layer Adaptive Instance Normalization for Fine-grained Style Similaritycited by 34
  25. 2021Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Modelcited by 34
  26. 2018The Shape of Art History in the Eyes of the Machinecited by 33
  27. 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33
  28. 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
  29. 2018Style and Content Disentanglement in Generative Adversarial Networkscited by 29
  30. 2018Weakly Supervised Object Detection in Artworkscited by 27
  31. 2019Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learningcited by 26
  32. 2021Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillationcited by 22
  33. 2019Context-Aware Embeddings for Automatic Art Analysiscited by 16
  34. 2020Measuring the Biases and Effectiveness of Content-Style Disentanglementcited by 14
  35. 2021Disentangled Representations from Non-Disentangled Modelscited by 13
  36. 2021GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graphcited by 11
  37. 2016Pandora: Description of a Painting Database for Art Movement Recognition with Baselines and Perspectivescited by 4
  38. 2020Improving Style-Content Disentanglement in Image-to-Image Translationcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 11; 2025: 6.

18 papers.

Sort:

By area:

  1. 2024StyleTokenizer: Defining Image Style by a Single Instance for Controlling Diffusion Modelscited by 32
  2. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  3. 2024Diffusion-Based Visual Art Creation: A Survey and New Perspectivescited by 17
  4. 2024Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedbackcited by 9
  5. 2024Have Large Vision-Language Models Mastered Art History?cited by 9
  6. 2024U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AIcited by 7
  7. 2024A Plug-and-Play Method for Guided Multi-contrast MRI Reconstruction based on Content/Style Modelingcited by 3
  8. 2024Latent Space Disentanglement in Diffusion Transformers Enables Precise Zero-shot Semantic Editingcited by 3
  9. 2023MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoisingcited by 2
  10. 2024Diff-PCC: Diffusion-based Neural Compression for 3D Point Cloudscited by 2
  11. 2024No Annotations for Object Detection in Art through Stable Diffusioncited by 2
  12. 2025CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Artcited by 2
6 more
  1. 2025The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Modelscited by 2
  2. 2024Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Fine-grained Semantic Editingcited by 1
  3. 2025MTADiffusion: Mask Text Alignment Diffusion Model for Object Inpaintingcited by 1
  4. 2025Identifying Prompted Artist Names from Generated Imagescited by 1
  5. 2025Instance-Level Generation for Representation Learningcited by 1
  6. 2025Language-Guided Trajectory Traversal in Disentangled Stable Diffusion Latent Space for Factorized Medical Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Silent Brush: Evaluating Artistic Style Leakage in AI Art Generation
  2. 2024WikiStyle+: A Multimodal Approach to Content-Style Representation Disentanglement for Artistic Image Stylization
  3. 2024Learning Artistic Signatures: Symmetry Discovery and Style Transfer
  4. 2022Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models
  5. 2024GalleryGPT: Analyzing Paintings with Large Multimodal Models
  6. 2026Style-Aware Gloss Control for Generative Non-Photorealistic Rendering
  7. 2025LouvreSAE: Sparse Autoencoders for Interpretable and Controllable Style Transfer
  8. 2024Measuring Style Similarity in Diffusion Models