mlsanityData to 7 Oct 2026

InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning

arXiv 2304.03411 ↗First posted 6 Apr 2023cs.CVcited by 343

Not on the map. Why?

Cited by 159 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in personalized image generation allow a pre-trained text-to-image model to learn a new concept from a set of images. However, existing personalization approaches usually require heavy test-time finetuning for each concept, which is time-consuming and difficult to scale. We propose InstantBooth, a novel approach built upon pre-trained text-to-image models that enables instant text-guided image personalization without any test-time finetuning. We achieve this with several major components. First, we learn the general concept of the input images by converting them to a textual token with a learnable image encoder. Second, to keep the fine details of the identity, we learn rich visual feature representation by introducing a few adapter layers to the pre-trained model. We train our components only on text-image pairs without using paired images of the same concept. Compared to test-time finetuning-based methods like DreamBooth and Textual-Inversion, our model can generate competitive results on unseen concepts concerning language-image alignment, image fidelity, and identity preservation while being 100 times faster.

Key facts. Cited by 343 papers in the map (data through 7 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

43 of 59 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
31 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  3. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  6. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  7. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  9. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  10. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  11. 2016Generative Adversarial Text to Image Synthesiscited by 735
  12. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  13. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  14. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  15. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  16. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  17. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  18. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  19. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  20. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  21. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  22. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  23. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  24. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  25. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
  26. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  27. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 116
  28. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  29. 2021Open-World Entity Segmentationcited by 74
  30. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generationcited by 60
  31. 2021PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistencycited by 59

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 76; 2024: 125; 2025: 107; 2026: 35.

343 papers.

Sort:

By area:

  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  2. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  3. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  4. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  5. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  6. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  7. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  8. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  9. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  10. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  11. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  12. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
38 more
  1. 2023Style Aligned Image Generation via Shared Attentioncited by 214
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  4. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  5. 2023AnyText: Multilingual Visual Text Generation And Editingcited by 165
  6. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  7. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  8. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  9. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  10. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  11. 2024Training-Free Consistent Text-to-Image Generationcited by 123
  12. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  13. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  14. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  15. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  16. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  17. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Modelscited by 86
  18. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  19. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  20. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  21. 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
  22. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 71
  23. 2023PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion Modelscited by 70
  24. 2023Orthogonal Adaptation for Modular Customization of Diffusion Modelscited by 70
  25. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  26. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  27. 2025Multi-subject Open-set Personalization in Video Generationcited by 65
  28. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Modelscited by 64
  29. 2023Inserting Anybody in Diffusion Models via Celeb Basiscited by 63
  30. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 62
  31. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  32. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  33. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 58
  34. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
  35. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  36. 2023EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methodscited by 52
  37. 2024JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generationcited by 52
  38. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51

50 most cited of 343 All 343 (JSON)