mlsanityData to 14 Oct 2026

Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning

arXiv 2307.11410 ↗First posted 21 Jul 2023cs.CVcited by 191

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Recent progress in personalized image generation using diffusion models has been significant. However, development in the area of open-domain and non-fine-tuning personalized image generation is proceeding rather slowly. In this paper, we propose Subject-Diffusion, a novel open-domain personalized image generation model that, in addition to not requiring test-time fine-tuning, also only requires a single reference image to support personalized generation of single- or multi-subject in any domain. Firstly, we construct an automatic data labeling tool and use the LAION-Aesthetics dataset to construct a large-scale dataset consisting of 76M images and their corresponding subject detection bounding boxes, segmentation masks and text descriptions. Secondly, we design a new unified framework that combines text and image semantics by incorporating coarse location and fine-grained reference image control to maximize subject fidelity and generalization. Furthermore, we also adopt an attention control mechanism to support multi-subject generation. Extensive qualitative and quantitative results demonstrate that our method outperforms other SOTA frameworks in single, multiple, and human customized image generation. Please refer to our hrefhttps://oppo-mente-lab.github.io/subjectdiffusion/project page

Key facts. Cited by 191 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 65 of its 70 references that are in the map. First posted on arXiv 21 Jul 2023.

Builds on

65 of 70 references.

Closest ancestors · 26

Same direction, newest first

  1. 2023Continuous Layout Editing of Single Images with Diffusion Models22 Jun
  2. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment15 Jun
  3. 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
  4. 2023Detector Guidance for Multi-Object Text-to-Image Generation4 Jun
  5. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  6. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models23 May
  7. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  8. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA12 Apr
  9. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis7 Apr
  10. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models5 Apr
  11. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  12. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation16 Mar
14 more
  1. 2023Highly Personalized Text Embedding for Image Manipulation by Stable Diffusion15 Mar
  2. 2023Cones: Concept Neurons in Diffusion Models for Customized Generation9 Mar
  3. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models23 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  7. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  9. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  10. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts27 Oct
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  13. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  14. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2023Segment Anythingcited by 8,234
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
27 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  3. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  4. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  5. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  6. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  10. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  11. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  12. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  13. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  14. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  15. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  16. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
  17. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  18. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 212
  19. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
  20. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  21. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  22. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  23. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  24. 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
  25. 2023Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approachcited by 43
  26. 2023Gradient-Free Textual Inversioncited by 29
  27. 2023Controllable Textual Inversion for Personalized Text-to-Image Generationcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 21; 2024: 87; 2025: 63; 2026: 20.

191 papers.

Sort:

By area:

  1. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  2. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  3. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  4. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 188
  5. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
  6. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  7. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  8. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  9. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  10. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 112
  11. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 109
  12. 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 105
38 more
  1. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generationcited by 93
  2. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Modelscited by 86
  3. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  4. 2025DreamO: A Unified Framework for Image Customizationcited by 83
  5. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
  6. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 77
  7. 2023PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion Modelscited by 70
  8. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 68
  9. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 60
  10. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 59
  11. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  12. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51
  13. 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
  14. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 47
  15. 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generationcited by 47
  16. 2024Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generationcited by 43
  17. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 43
  18. 2024Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigmcited by 39
  19. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  20. 2025ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Modelscited by 38
  21. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  22. 2024IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Modelscited by 32
  23. 2024SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Controlcited by 32
  24. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  25. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  26. 2024GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Modelscited by 29
  27. 2024StoryMaker: Towards Holistic Consistent Characters in Text-to-image Generationcited by 29
  28. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  29. 2024PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradationcited by 29
  30. 2024MC2: Multi-concept Guidance for Customized Multi-concept Generationcited by 26
  31. 2023CustomNet: Zero-shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Modelscited by 25
  32. 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25
  33. 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 25
  34. 2024CPR: Retrieval Augmented Generation for Copyright Protectioncited by 24
  35. 2025Personalize Anything for Free with Diffusion Transformercited by 23
  36. 2023High-fidelity Person-centric Subject-to-Image Synthesiscited by 21
  37. 2023MagDiff: Multi-Alignment Diffusion for High-Fidelity Video Generation and Editingcited by 21
  38. 2024DisenStudio: Customized Multi-subject Text-to-Video Generation with Disentangled Spatial Controlcited by 20

50 most cited of 191 All 191 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025IMAGE-ALCHEMY: Advancing subject fidelity in personalised text-to-image generation
  2. 2024Fast Personalized Text-to-Image Syntheses With Attention Injection
  3. 2024JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation
  4. 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
  5. 2026CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets
  6. 2025SceneBooth: Diffusion-based Framework for Subject-preserved Text-to-Image Generation
  7. 2024DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Models
  8. 2025PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards