mlsanityData to 7 Oct 2026

OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models

arXiv 2403.10983 ↗First posted 16 Mar 2024cs.CVcited by 64

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Personalization is an important topic in text-to-image generation, especially the challenging multi-concept personalization. Current multi-concept methods are struggling with identity preservation, occlusion, and the harmony between foreground and background. In this work, we propose OMG, an occlusion-friendly personalized generation framework designed to seamlessly integrate multiple concepts within a single image. We propose a novel two-stage sampling solution. The first stage takes charge of layout generation and visual comprehension information collection for handling occlusions. The second one utilizes the acquired visual comprehension information and the designed noise blending to integrate multiple concepts while considering occlusions. We also observe that the initiation denoising timestep for noise blending is the key to identity preservation and layout. Moreover, our method can be combined with various single-concept models, such as LoRA and InstantID without additional tuning. Especially, LoRA models on civitai.com can be exploited directly. Extensive experiments demonstrate that OMG exhibits superior performance in multi-concept personalization.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 46 of its 50 references that are in the map. First posted on arXiv 16 Mar 2024.

Builds on

46 of 50 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  2. 2023Cross Initialization for Personalized Text-to-Image Generation26 Dec
  3. 2023Compositional Inversion for Stable Diffusion Models13 Dec
  4. 2023Orthogonal Adaptation for Modular Customization of Diffusion Models5 Dec
  5. 2023CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization24 Nov
  6. 2023Multi-Concept T2I-Zero: Tweaking Only The Text Embeddings and Nothing Else11 Oct
  7. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  8. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models25 May
  9. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  10. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA12 Apr
  11. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  12. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation16 Mar
5 more
  1. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models23 Feb
  2. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation16 Feb
  3. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2023Segment Anythingcited by 8,158
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  10. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  11. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  12. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
17 more
  1. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  2. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  3. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  4. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  5. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  6. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  7. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  8. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  9. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  10. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  11. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 71
  12. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  13. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  14. 2023Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approachcited by 43
  15. 2023A Data Perspective on Enhanced Identity Preservation for Diffusion Personalizationcited by 18
  16. 2023Customization Assistant for Text-to-image Generationcited by 11
  17. 2023InstructBooth: Instruction-following Personalized Text-to-Image Generationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 35; 2026: 6.

64 papers.

Sort:

By area:

  1. 2024A Survey on LoRA of Large Language Modelscited by 67
  2. 2025Let Them Talk: Audio-Driven Multi-Person Conversational Video Generationcited by 64
  3. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
  4. 2025TokenVerse: Versatile Multi-concept Personalization in Token Modulation Spacecited by 51
  5. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 46
  6. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  7. 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?cited by 32
  8. 2024InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generationcited by 30
  9. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  10. 2025Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Promptscited by 28
  11. 2024StoryMaker: Towards Holistic Consistent Characters in Text-to-image Generationcited by 27
  12. 2025Personalize Anything for Free with Diffusion Transformercited by 22
38 more
  1. 2024LoRACLR: Contrastive Adaptation for Customization of Diffusion Modelscited by 20
  2. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  3. 2024DIVE: Taming DINO for Subject-Driven Video Editingcited by 17
  4. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  5. 2024Towards Multiple Character Image Animation Through Enhancing Implicit Decouplingcited by 15
  6. 2025Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adaptercited by 13
  7. 2025LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformerscited by 12
  8. 2024ID-Patch: Robust ID Association for Group Photo Personalizationcited by 11
  9. 2024AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generationcited by 10
  10. 2025DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editabilitycited by 10
  11. 2025LatexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blendingcited by 10
  12. 2024Contrastive Test-Time Composition of Multiple LoRA Models for Image Generationcited by 9
  13. 2025Consistent Image Layout Editing with Diffusion Modelscited by 9
  14. 2024Storynizor: Consistent Story Generation via Inter-Frame Synchronized and Shuffled ID Injectioncited by 8
  15. 2025MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humanscited by 8
  16. 2024An Efficient and Harmonized Framework for Balanced Cross-Domain Feature Integrationcited by 7
  17. 2025ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generationcited by 7
  18. 2024Character-Adapter: Prompt-Guided Region Control for High-Fidelity Character Customizationcited by 6
  19. 2025Piece it Together: Part-Based Concepting with IP-Priorscited by 6
  20. 2025IdentityStory: Taming Your Identity-Preserving Generator for Human-Centric Story Generationcited by 6
  21. 2024CAT: Contrastive Adapter Training for Personalized Image Generationcited by 5
  22. 2024DreamSteerer: Enhancing Source Image Conditioned Editability using Personalized Diffusion Modelscited by 5
  23. 2025DAM-VSR: Disentanglement of Appearance and Motion for Video Super-Resolutioncited by 5
  24. 2025Canvas-to-Image: Compositional Image Generation with Multimodal Controlscited by 5
  25. 2025Virtually Being: Customizing Camera-Controllable Video Diffusion Models with Multi-View Performance Capturescited by 4
  26. 2024Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attentioncited by 3
  27. 2025IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Promptingcited by 3
  28. 2025Continual Personalization for Diffusion Modelscited by 3
  29. 2025CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvascited by 3
  30. 2024Foundation Cures Personalization: Improving Personalized Models' Prompt Consistency via Hidden Foundation Knowledgecited by 2
  31. 2025Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusioncited by 2
  32. 2025ShowFlow: From Robust Single Concept to Condition-Free Multi-Concept Generationcited by 2
  33. 2025Category-Aware 3D Object Composition with Disentangled Texture and Shape Multi-view Diffusioncited by 2
  34. 2025Composing Concepts from Images and Videos via Concept-prompt Bindingcited by 2
  35. 2024SPDiffusion: Semantic Protection Diffusion Models for Multi-concept Text-to-image Generationcited by 1
  36. 2024PersonaCraft: Personalized and Controllable Full-Body Multi-Human Scene Generation Using Occlusion-Aware 3D-Conditioned Diffusioncited by 1
  37. 2025Human Preference-Aligned Concept Customization Benchmark via Decomposed Evaluationcited by 1
  38. 2025Griffin: Generative Reference and Layout Guided Image Compositioncited by 1

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
  2. 2024ComFusion: Personalized Subject Generation in Multiple Specific Scenes From Single Image
  3. 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
  4. 2024Isolated Diffusion: Optimizing Multi-Concept Text-to-Image Generation Training-Freely with Isolated Diffusion Guidance
  5. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  6. 2025Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects
  7. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
  8. 2024TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generation