mlsanityData to 7 Oct 2026

Inversion-Based Style Transfer with Diffusion Models

arXiv 2211.13203 ↗First posted 23 Nov 2022cs.CV cs.GRcited by 290

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

The artistic style within a painting is the means of expression, which includes not only the painting material, colors, and brushstrokes, but also the high-level attributes including semantic elements, object shapes, etc. Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. The pre-trained text-to-image synthesis diffusion probabilistic models have achieved remarkable quality, but it often requires extensive textual descriptions to accurately portray attributes of a particular painting. We believe that the uniqueness of an artwork lies precisely in the fact that it cannot be adequately explained with normal language. Our key idea is to learn artistic style directly from a single painting and then guide the synthesis without providing complex textual descriptions. Specifically, we assume style as a learnable textual description of a painting. We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image, thus capturing and transferring the artistic style of a painting. We demonstrate the quality and efficiency of our method on numerous paintings of various artists and styles. Code and models are available at https://github.com/zyxElsa/InST.

Key facts. Cited by 290 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 47 of its 60 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

47 of 60 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  2. 2022Creative Painting with Latent Diffusion Models29 Sep
  3. 2022Draw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion27 Sep
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
28 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  2. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  3. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  4. 2017Universal Style Transfer via Feature Transformscited by 352
  5. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  6. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  7. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  8. 2018Arbitrary Style Transfer with Style-Attentional Networkscited by 197
  9. 2021StyTr2: Image Style Transfer with Transformerscited by 186
  10. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  11. 2021AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfercited by 178
  12. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 167
  13. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  14. 2017Visual Attribute Transfer through Deep Image Analogycited by 123
  15. 2022Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalizationcited by 122
  16. 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
  17. 2016Controlling Perceptual Factors in Neural Style Transfercited by 99
  18. 2022Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learningcited by 93
  19. 2020Arbitrary Video Style Transfer via Multi-Channel Correlationcited by 84
  20. 2020Arbitrary Style Transfer via Multi-Adaptation Networkcited by 70
  21. 2022DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylizationcited by 45
  22. 2021Image-Based CLIP-Guided Essence Transfercited by 39
  23. 2019Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfercited by 36
  24. 2021Language-Driven Image Style Transfercited by 36
  25. 2022StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translationcited by 28
  26. 2020xCos: An Explainable Cosine Metric for Face Verification Taskcited by 24
  27. 2022Name Your Style: An Arbitrary Artist-aware Image Style Transfercited by 14
  28. 2020Distribution Aligned Multimodal and Multi-Domain Image Stylizationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 44; 2024: 96; 2025: 100; 2026: 50.

290 papers.

Sort:

By area:

  1. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylizationcited by 251
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 197
  4. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 194
  5. 2023StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-Oncited by 173
  6. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  7. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  8. 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 100
  9. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
  10. 2024CSGO: Content-Style Composition in Text-to-Image Generationcited by 76
  11. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75
  12. 2023FocalDreamer: Text-driven 3D Editing via Focal-fusion Assemblycited by 75
38 more
  1. 2024An Undetectable Watermark for Generative Image Modelscited by 74
  2. 2024DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representationscited by 68
  3. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  4. 2025Insert Anything: Image Insertion via In-Context Editing in DiTcited by 61
  5. 2024StyleShot: A Snapshot on Any Stylecited by 60
  6. 2024CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Modelcited by 54
  7. 2025K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAscited by 48
  8. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  9. 2023On Exact Inversion of DPM-Solverscited by 47
  10. 2024Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Taskscited by 47
  11. 2024StyleMaster: Stylize Your Video with Artistic Generation and Translationcited by 47
  12. 2023WaveDM: Wavelet-Based Diffusion Models for Image Restorationcited by 46
  13. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  14. 2023ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priorscited by 40
  15. 2023StyleAdapter: A Unified Stylized Image Generation Modelcited by 39
  16. 2023ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptationcited by 39
  17. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  18. 2023Real-World Image Variation by Aligning Diffusion Inversion Chaincited by 37
  19. 2023Exploiting Diffusion Prior for Generalizable Dense Predictioncited by 37
  20. 2023Relation-Aware Diffusion Model for Controllable Poster Layout Generationcited by 36
  21. 2024FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instructioncited by 35
  22. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  23. 2024Text-to-Vector Generation with Neural Path Representationcited by 32
  24. 2024StyleTokenizer: Defining Image Style by a Single Instance for Controlling Diffusion Modelscited by 32
  25. 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?cited by 32
  26. 2025Attention Distillation: A Unified Approach to Visual Characteristics Transfercited by 32
  27. 2023General Image-to-Image Translation with One-Shot Image Guidancecited by 31
  28. 2024DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protectioncited by 31
  29. 2025How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Modelscited by 31
  30. 2025OmniStyle: Filtering High Quality Style Transfer Data at Scalecited by 30
  31. 2023BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeyscited by 29
  32. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  33. 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
  34. 2024StyleBooth: Image Style Editing with Multimodal Instructioncited by 28
  35. 2024BootPIG: Bootstrapping Zero-shot Personalized Image Generation Capabilities in Pretrained Diffusion Modelscited by 27
  36. 2023DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Modelscited by 26
  37. 2024StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elementscited by 26
  38. 2023Diff-Privacy: Diffusion-based Face Privacy Protectioncited by 25

50 most cited of 290 All 290 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Highly Realistic Artistic Style Transfer via Stable Diffusion with Step-aware and Layer-aware Prompt
  2. 2023ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank
  3. 2025Style Composition within Distinct LoRA modules for Traditional Art
  4. 2025The Cow of Rembrandt - Analyzing Artistic Prompt Interpretation in Text-to-Image Models
  5. 2024Learning Artistic Signatures: Symmetry Discovery and Style Transfer
  6. 2023ArtFusion: Controllable Arbitrary Style Transfer using Dual Conditional Latent Diffusion Models
  7. 2025Loomis Painter: Reconstructing the Painting Process
  8. 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis