mlsanityData to 14 Oct 2026

SVDiff: Compact Parameter Space for Diffusion Fine-Tuning

arXiv 2303.11305 ↗First posted 20 Mar 2023cs.CVcited by 338

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have achieved remarkable success in text-to-image generation, enabling the creation of high-quality images from text prompts or other modalities. However, existing methods for customizing these models are limited by handling multiple personalized subjects and the risk of overfitting. Moreover, their large number of parameters is inefficient for model storage. In this paper, we propose a novel approach to address these limitations in existing text-to-image diffusion models for personalization. Our method involves fine-tuning the singular values of the weight matrices, leading to a compact and efficient parameter space that reduces the risk of overfitting and language drifting. We also propose a Cut-Mix-Unmix data-augmentation technique to enhance the quality of multi-subject image generation and a simple text-based image editing framework. Our proposed SVDiff method has a significantly smaller model size compared to existing methods (approximately 2,200 times fewer parameters compared with vanilla DreamBooth), making it more practical for real-world applications.

Key facts. Cited by 338 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 64 of its 87 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

64 of 87 references.

Closest ancestors · 21

Same direction, newest first

  1. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Models14 Mar
  2. 2023Cones: Concept Neurons in Diffusion Models for Customized Generation9 Mar
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  6. 2022The Stable Artist: Steering Semantics in Diffusion Latent Space12 Dec
  7. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  8. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Models8 Dec
  9. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  11. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  12. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
9 more
  1. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  3. 2022Blended Latent Diffusion6 Jun
  4. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  7. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translation16 Mar
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  9. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
31 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  6. 2023Consistency Modelscited by 1,822
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2022Editing Models with Task Arithmeticcited by 1,120
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  10. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  11. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  13. 2017Learning multiple visual domains with residual adapterscited by 557
  14. 2023Zero-shot Image-to-Image Translationcited by 505
  15. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  16. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  17. 2023Universal Guidance for Diffusion Modelscited by 437
  18. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  19. 2020Closed-Form Factorization of Latent Semantics in GANscited by 408
  20. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 304
  21. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
  22. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  23. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 187
  24. 2020Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANscited by 132
  25. 2020Rewriting a Deep Generative Modelcited by 107
  26. 2019Image Generation From Small Datasets via Batch Statistics Adaptationcited by 102
  27. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 67
  28. 2020Few-Shot Adaptation of Generative Adversarial Networkscited by 64
  29. 2023Word-As-Image for Semantic Typographycited by 63
  30. 2022Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuningcited by 50
  31. 2020Navigating the GAN Parameter Space for Semantic Image Editingcited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 63; 2024: 131; 2025: 114; 2026: 30.

338 papers.

Sort:

By area:

  1. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  2. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  3. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  4. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
  5. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  6. 2023Style Aligned Image Generation via Shared Attentioncited by 216
  7. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
  8. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  9. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  10. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 188
  11. 2024Mixture of LoRA Expertscited by 181
  12. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 144
38 more
  1. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  2. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  3. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  4. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  5. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  6. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 109
  7. 2024Improving Diffusion Models for Authentic Virtual Try-on in the Wildcited by 109
  8. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  9. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Modelscited by 86
  10. 2024Asymmetry in Low-Rank Adapters of Foundation Modelscited by 86
  11. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  12. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 85
  13. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Modelscited by 83
  14. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  15. 2023ReVersion: Diffusion-Based Relation Inversion from Imagescited by 82
  16. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 82
  17. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editingcited by 76
  18. 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
  19. 2023Orthogonal Adaptation for Modular Customization of Diffusion Modelscited by 70
  20. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 68
  21. 2025Multi-subject Open-set Personalization in Video Generationcited by 66
  22. 2024LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insightscited by 65
  23. 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ballcited by 64
  24. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Modelscited by 64
  25. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 63
  26. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  27. 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Modelcited by 60
  28. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  29. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 60
  30. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 59
  31. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 58
  32. 2024SVFT: Parameter-Efficient Fine-Tuning with Singular Vectorscited by 58
  33. 2023Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunitycited by 56
  34. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
  35. 2024JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generationcited by 52
  36. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51
  37. 2024BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Modelscited by 50
  38. 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 50

50 most cited of 338 All 338 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models
  2. 2024DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Models
  3. 2025Steering Guidance for Personalized Text-to-Image Diffusion Models
  4. 2025Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization
  5. 2024Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models
  6. 2024An Improved Method for Personalizing Diffusion Models
  7. 2023PRedItOR: Text Guided Image Editing with Diffusion Prior
  8. 2023HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models