# Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis

arXiv: https://arxiv.org/abs/2304.10278 · HTML: https://mlsanity.com/paper/2304.10278/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 56 of its 90 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 56)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [PointCLIP: Point Cloud Understanding by CLIP](https://mlsanity.com/paper/2112.02413/) (2021)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature](https://mlsanity.com/paper/1505.00855/) (2015)
- [Fake it till you make it: Learning transferable representations from synthetic ImageNet clones](https://mlsanity.com/paper/2212.08420/) (2022)
- [DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis](https://mlsanity.com/paper/2008.05865/) (2020)
- [Diffusion-based Image Translation using Disentangled Style and Content Representation](https://mlsanity.com/paper/2209.15264/) (2022)
- [Unsupervised Learning of Disentangled Representations from Video](https://mlsanity.com/paper/1705.10915/) (2017)

## Built on by (most cited first, 18 of 18)

- [StyleTokenizer: Defining Image Style by a Single Instance for Controlling Diffusion Models](https://mlsanity.com/paper/2409.02543/) (2024)
- [Would Deep Generative Models Amplify Bias in Future Models?](https://mlsanity.com/paper/2404.03242/) (2024)
- [Diffusion-Based Visual Art Creation: A Survey and New Perspectives](https://mlsanity.com/paper/2408.12128/) (2024)
- [Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedback](https://mlsanity.com/paper/2402.02346/) (2024)
- [Have Large Vision-Language Models Mastered Art History?](https://mlsanity.com/paper/2409.03521/) (2024)
- [U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI](https://mlsanity.com/paper/2406.15386/) (2024)
- [A Plug-and-Play Method for Guided Multi-contrast MRI Reconstruction based on Content/Style Modeling](https://mlsanity.com/paper/2409.13477/) (2024)
- [Latent Space Disentanglement in Diffusion Transformers Enables Precise Zero-shot Semantic Editing](https://mlsanity.com/paper/2411.08196/) (2024)
- [MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising](https://mlsanity.com/paper/2312.10899/) (2023)
- [Diff-PCC: Diffusion-based Neural Compression for 3D Point Clouds](https://mlsanity.com/paper/2408.10543/) (2024)
- [No Annotations for Object Detection in Art through Stable Diffusion](https://mlsanity.com/paper/2412.06286/) (2024)
- [CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art](https://mlsanity.com/paper/2503.12018/) (2025)
- [The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Models](https://mlsanity.com/paper/2507.23313/) (2025)
- [Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Fine-grained Semantic Editing](https://mlsanity.com/paper/2408.13335/) (2024)
- [MTADiffusion: Mask Text Alignment Diffusion Model for Object Inpainting](https://mlsanity.com/paper/2506.23482/) (2025)
- [Identifying Prompted Artist Names from Generated Images](https://mlsanity.com/paper/2507.18633/) (2025)
- [Instance-Level Generation for Representation Learning](https://mlsanity.com/paper/2510.09171/) (2025)
- [Language-Guided Trajectory Traversal in Disentangled Stable Diffusion Latent Space for Factorized Medical Image Generation](https://mlsanity.com/paper/2503.23623/) (2025)

## Abstract (arXiv)

The duality of content and style is inherent to the nature of art. For humans, these two elements are clearly different: content refers to the objects and concepts in the piece of art, and style to the way it is expressed. This duality poses an important challenge for computer vision. The visual appearance of objects and concepts is modulated by the style that may reflect the author's emotions, social trends, artistic movement, etc., and their deep comprehension undoubtfully requires to handle both. A promising step towards a general paradigm for art analysis is to disentangle content and style, whereas relying on human annotations to cull a single aspect of artworks has limitations in learning semantic concepts and the visual appearance of paintings. We thus present GOYA, a method that distills the artistic knowledge captured in a recent generative model to disentangle content and style. Experiments show that synthetically generated images sufficiently serve as a proxy of the real distribution of artworks, allowing GOYA to separately represent the two elements of art while keeping more information than existing methods.