mlsanityData to 7 Oct 2026

Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models

arXiv 2311.12092 ↗First posted 20 Nov 2023cs.CVcited by 148

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We present a method to create interpretable concept sliders that enable precise control over attributes in image generations from diffusion models. Our approach identifies a low-rank parameter direction corresponding to one concept while minimizing interference with other attributes. A slider is created using a small set of prompts or sample images; thus slider directions can be created for either textual or visual concepts. Concept Sliders are plug-and-play: they can be composed efficiently and continuously modulated, enabling precise control over image generation. In quantitative experiments comparing to previous editing techniques, our sliders exhibit stronger targeted edits with lower interference. We showcase sliders for weather, age, styles, and expressions, as well as slider compositions. We show how sliders can transfer latents from StyleGAN for intuitive editing of visual concepts for which textual description is difficult. We also find that our method can help address persistent quality issues in Stable Diffusion XL including repair of object deformations and fixing distorted hands. Our code, data, and trained sliders are available at https://sliders.baulab.info/

Key facts. Cited by 148 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 37 of its 45 references that are in the map. First posted on arXiv 20 Nov 2023.

Builds on

37 of 45 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Models14 Mar
  2. 2022Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models16 Dec
  3. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  5. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun

Other references · 30

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  9. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2021Alias-Free Generative Adversarial Networkscited by 1,181
18 more
  1. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  2. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  3. 2023Erasing Concepts from Diffusion Modelscited by 530
  4. 2023Zero-shot Image-to-Image Translationcited by 503
  5. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  6. 2023Unified Concept Editing in Diffusion Modelscited by 377
  7. 2022Understanding Diffusion Models: A Unified Perspectivecited by 337
  8. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 331
  9. 2022Diffusion Models already have a Semantic Latent Spacecited by 324
  10. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  11. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  12. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  13. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  14. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  15. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometrycited by 121
  16. 2023Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Modelscited by 60
  17. 2023Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Modelscited by 41
  18. 2023Unsupervised Discovery of Semantic Latent Directions in Diffusion Modelscited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 41; 2025: 58; 2026: 47.

148 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  3. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  4. 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Modelscited by 46
  5. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  6. 2024Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directionscited by 39
  7. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  8. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  9. 2024Interpreting the Weight Space of Customized Diffusion Modelscited by 35
  10. 2025Efficient Diffusion Models: A Surveycited by 35
  11. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  12. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 33
38 more
  1. 2024One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Modelscited by 30
  2. 2025Diff2Flow: Training Flow Matching Models via Diffusion Model Alignmentcited by 27
  3. 2024Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesiscited by 26
  4. 2025RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformerscited by 26
  5. 2024FluxSpace: Disentangled Semantic Editing in Rectified Flow Transformerscited by 25
  6. 2025SliderSpace: Decomposing the Visual Capabilities of Diffusion Modelscited by 24
  7. 2024TexSliders: Diffusion-Based Texture Editing in CLIP Spacecited by 23
  8. 2024Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmarkcited by 21
  9. 2025Distilling Diversity and Control in Diffusion Modelscited by 21
  10. 2024PreciseControl: Enhancing Text-To-Image Diffusion Models with Fine-Grained Attribute Controlcited by 20
  11. 2024SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generationcited by 19
  12. 2024Stylus: Automatic Adapter Selection for Diffusion Modelscited by 18
  13. 2024Learning Discrete Concepts in Latent Hierarchical Modelscited by 17
  14. 2024FouRA: Fourier Low Rank Adaptationcited by 15
  15. 2025Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editingcited by 15
  16. 2024Prompt Sliders for Fine-Grained Control, Editing and Erasing of Concepts in Diffusion Modelscited by 14
  17. 2025When Are Concepts Erased From Diffusion Models?cited by 14
  18. 2024Giving a Hand to Diffusion Models: a Two-Stage Approach to Improving Conditional Human Image Generationcited by 13
  19. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  20. 2025Dual-Process Image Generationcited by 12
  21. 2025UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedbackcited by 12
  22. 2025SliderEdit: Continuous Image Editing with Fine-Grained Instruction Controlcited by 12
  23. 2025Interpretable Generative Models through Post-hoc Concept Bottleneckscited by 11
  24. 2024PaRa: Personalizing Text-to-Image Diffusion via Parameter Rank Reductioncited by 10
  25. 2025HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait Synthesiscited by 10
  26. 2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 10
  27. 2024Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy?cited by 9
  28. 2024Contrastive Test-Time Composition of Multiple LoRA Models for Image Generationcited by 9
  29. 2024RHanDS: Refining Malformed Hands for Generated Images with Decoupled Structure and Style Guidancecited by 9
  30. 2025MARBLE: Material Recomposition and Blending in CLIP-Spacecited by 9
  31. 2024On Fairness of Low-Rank Adaptation of Large Modelscited by 8
  32. 2024FairGen: Enhancing Fairness in Text-to-Image Diffusion Models via Self-Discovering Latent Directionscited by 8
  33. 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editingcited by 8
  34. 2025SAEdit: Token-level control for continuous image editing via Sparse AutoEncodercited by 8
  35. 2025Group Relative Attention Guidance for Image Editingcited by 8
  36. 2025A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMscited by 7
  37. 2025PractiLight: Practical Light Control Using Foundational Diffusion Modelscited by 7
  38. 2024Learn Once, Edit Anywhere: Visual Direction Transfer for Diffusion Modelscited by 6

50 most cited of 148 All 148 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025All-in-One Slider for Attribute Manipulation in Diffusion Models
  2. 2025ACE: Concept Editing in Diffusion Models without Performance Degradation
  3. 2023The Hidden Language of Diffusion Models
  4. 2025DyME: Dynamic Multi-Concept Erasure in Diffusion Models with Bi-Level Orthogonal LoRA Adaptation
  5. 2026Training-Free Multi-Concept Image Editing
  6. 2025T-LoRA: Single Image Diffusion Model Customization Without Overfitting
  7. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  8. 2025Glance: Accelerating Diffusion Models with 1 Sample