mlsanityData to 7 Oct 2026

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

arXiv 2208.12242 ↗First posted 25 Aug 2022cs.CV cs.GR cs.LGcited by 3,286

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Large text-to-image models achieved a remarkable leap in the evolution of AI, enabling high-quality and diverse synthesis of images from a given text prompt. However, these models lack the ability to mimic the appearance of subjects in a given reference set and synthesize novel renditions of them in different contexts. In this work, we present a new approach for "personalization" of text-to-image diffusion models. Given as input just a few images of a subject, we fine-tune a pretrained text-to-image model such that it learns to bind a unique identifier with that specific subject. Once the subject is embedded in the output domain of the model, the unique identifier can be used to synthesize novel photorealistic images of the subject contextualized in different scenes. By leveraging the semantic prior embedded in the model with a new autogenous class-specific prior preservation loss, our technique enables synthesizing the subject in diverse scenes, poses, views and lighting conditions that do not appear in the reference images. We apply our technique to several previously-unassailable tasks, including subject recontextualization, text-guided view synthesis, and artistic rendering, all while preserving the subject's key features. We also provide a new dataset and evaluation protocol for this new task of subject-driven generation. Project page: https://dreambooth.github.io/

Key facts. Cited by 3,286 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 66 of its 78 references that are in the map. First posted on arXiv 25 Aug 2022.

Builds on

66 of 78 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022Blended Latent Diffusion6 Jun
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  5. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance10 Dec
  6. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  7. 2021Palette: Image-to-Image Diffusion Models10 Nov
  8. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
38 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  6. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  12. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  13. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  14. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  15. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  16. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  17. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  18. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  19. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  20. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  21. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  22. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  23. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  24. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  25. 2021Few-shot Image Generation via Cross-domain Correspondencecited by 206
  26. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  27. 2019Controllable Text-to-Image Generationcited by 187
  28. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  29. 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 134
  30. 2020Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANscited by 132
  31. 2020Few-shot Image Generation with Elastic Weight Consolidationcited by 122
  32. 2021Instance-Conditioned GANcited by 119
  33. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  34. 2021Paint by Wordcited by 107
  35. 2019MineGAN: effective knowledge transfer from GANs to target domains with few imagescited by 106
  36. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  37. 2019Semantic Object Accuracy for Generative Text-to-Image Synthesiscited by 89
  38. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 65; 2023: 732; 2024: 1084; 2025: 949; 2026: 456.

3,286 papers.

Sort:

By area:

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  7. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  10. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  11. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  12. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
38 more
  1. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  2. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  3. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  4. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  7. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  8. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  9. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  10. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  11. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  12. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  13. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  14. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  15. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  16. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  17. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  18. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  19. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  20. 2023Erasing Concepts from Diffusion Modelscited by 530
  21. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  22. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  23. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  24. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  25. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  26. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  27. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  28. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  29. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  30. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  31. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  32. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  33. 2024OmniGen: Unified Image Generationcited by 388
  34. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  35. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  36. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  37. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  38. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364

50 most cited of 3,286 All 3,286 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SceneBooth: Diffusion-based Framework for Subject-preserved Text-to-Image Generation
  2. 2024From Text to Pose to Image: Improving Diffusion Model Control and Quality
  3. 2022Implementing and Experimenting with Diffusion Models for Text-to-Image Generation
  4. 2026CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets
  5. 2024Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models
  6. 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
  7. 2023Synthetic Data from Diffusion Models Improves ImageNet Classification
  8. 2026Text-to-Image Models Need Less from Text Encoders Than You Think