mlsanityData to 7 Oct 2026

The Stable Artist: Steering Semantics in Diffusion Latent Space

arXiv 2212.06013 ↗First posted 12 Dec 2022cs.CV cs.AI cs.LGcited by 23

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Large, text-conditioned generative diffusion models have recently gained a lot of attention for their impressive performance in generating high-fidelity images from text alone. However, achieving high-quality results is almost unfeasible in a one-shot fashion. On the contrary, text-guided image generation involves the user making many slight changes to inputs in order to iteratively carve out the envisioned image. However, slight changes to the input prompt often lead to entirely different images being generated, and thus the control of the artist is limited in its granularity. To provide flexibility, we present the Stable Artist, an image editing approach enabling fine-grained control of the image generation process. The main component is semantic guidance (SEGA) which steers the diffusion process along variable numbers of semantic directions. This allows for subtle edits to images, changes in composition and style, as well as optimization of the overall artistic conception. Furthermore, SEGA enables probing of latent spaces to gain insights into the representation of concepts learned by the model, even complex ones such as 'carbon emission'. We demonstrate the Stable Artist on several tasks, showcasing high-quality image editing and composition.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 11 of its 14 references that are in the map. First posted on arXiv 12 Dec 2022.

Builds on

11 of 14 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 7

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  6. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  7. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 8; 2025: 4; 2026: 1.

23 papers.

Sort:

By area:

  1. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  4. 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 100
  5. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  6. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  7. 2023The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusioncited by 41
  8. 2023Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Modelscited by 36
  9. 2023Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Modelscited by 26
  10. 2023MDP: A Generalized Framework for Text-Guided Image Editing by Manipulating the Diffusion Pathcited by 18
  11. 2023MaskDiffusion: Boosting Text-to-Image Consistency with Conditional Maskcited by 17
  12. 2024Diffusion-Based Visual Art Creation: A Survey and New Perspectivescited by 17
11 more
  1. 2023Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generationcited by 12
  2. 2024Concept Arithmetics for Circumventing Concept Inhibition in Diffusion Modelscited by 11
  3. 2025Video Unlearning via Low-Rank Refusal Vectorcited by 11
  4. 2024Safeguard Text-to-Image Diffusion Models with Human Feedback Inversioncited by 10
  5. 2023Data Redaction from Conditional Generative Modelscited by 8
  6. 2023Zero-Shot Image Harmonization with Generative Model Priorcited by 8
  7. 2025Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generationscited by 6
  8. 2025Plug-and-Play Interpretable Responsible Text-to-Image Generation via Dual-Space Multi-facet Concept Controlcited by 5
  9. 2025Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generationcited by 2
  10. 2024"My Kind of Woman": Analysing Gender Stereotypes in AI through The Averageness Theory and EU Lawcited by 1
  11. 2026SHIFT: Steering Hidden Intermediates in Flow Transformerscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023SEGA: Instructing Text-to-Image Models using Semantic Guidance
  2. 2024OneActor: Consistent Character Generation via Cluster-Conditioned Guidance
  3. 2023Stable Diffusion Reference Only: Image Prompt and Blueprint Jointly Guided Multi-Condition Diffusion Model for Secondary Painting
  4. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance
  5. 2022High-Fidelity Guided Image Synthesis with Latent Diffusion Models
  6. 2024KnobGen: Controlling the Sophistication of Artwork in Sketch-Based Diffusion Models
  7. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs
  8. 2023Exploring Compositional Visual Generation with Latent Classifier Guidance