mlsanityData to 7 Oct 2026

UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalization

arXiv 2408.05939 ↗First posted 12 Aug 2024cs.CVcited by 46

Not on the map. Why?

Abstract (arXiv)

This paper presents UniPortrait, an innovative human image personalization framework that unifies single- and multi-ID customization with high face fidelity, extensive facial editability, free-form input description, and diverse layout generation. UniPortrait consists of only two plug-and-play modules: an ID embedding module and an ID routing module. The ID embedding module extracts versatile editable facial features with a decoupling strategy for each ID and embeds them into the context space of diffusion models. The ID routing module then combines and distributes these embeddings adaptively to their respective regions within the synthesized image, achieving the customization of single and multiple IDs. With a carefully designed two-stage training scheme, UniPortrait achieves superior performance in both single- and multi-ID customization. Quantitative and qualitative experiments demonstrate the advantages of our method over existing approaches as well as its good scalability, e.g., the universal compatibility with existing generative control tools. The project page is at https://aigcdesigngroup.github.io/UniPortrait-Page/ .

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 56 of its 68 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

56 of 68 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2022Generative Adversarial Networkscited by 5,603
38 more
  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  12. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  13. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  14. 2016Deep Networks with Stochastic Depthcited by 973
  15. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  16. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  17. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  18. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  19. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  20. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  21. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  22. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  23. 2020CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognitioncited by 320
  24. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  25. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  26. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  27. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  28. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  29. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  30. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  31. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 143
  32. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  33. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
  34. 2024Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generationcited by 68
  35. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Modelscited by 64
  36. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  37. 2024MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generationcited by 51
  38. 2024Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigmcited by 38

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 31; 2026: 8.

46 papers.

Sort:

By area:

  1. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  2. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  3. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  4. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  5. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  6. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 57
  7. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  8. 2025Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Promptscited by 28
  9. 2024StoryMaker: Towards Holistic Consistent Characters in Text-to-image Generationcited by 27
  10. 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
  11. 2024IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generationcited by 21
  12. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
34 more
  1. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  2. 2025WithAnyone: Towards Controllable and ID Consistent Image Generationcited by 16
  3. 2025Ingredients: Blending Custom Photos with Video Diffusion Transformerscited by 13
  4. 2025Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attackscited by 13
  5. 2025RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Modelscited by 13
  6. 2024ID-Patch: Robust ID Association for Group Photo Personalizationcited by 11
  7. 2024Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlookcited by 10
  8. 2025DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editabilitycited by 10
  9. 2024Text-to-Image Synthesis: A Decade Surveycited by 9
  10. 2025MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humanscited by 8
  11. 2025ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generationcited by 7
  12. 2025The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignmentcited by 5
  13. 2025Canvas-to-Image: Compositional Image Generation with Multimodal Controlscited by 5
  14. 2025A Comprehensive Survey on Visual Concept Mining in Text-to-image Diffusion Modelscited by 4
  15. 2025CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvascited by 3
  16. 2025DVI: Disentangling Semantic and Visual Identity for Training-Free Personalized Generationcited by 3
  17. 2026FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generationcited by 3
  18. 2025SubZero: Composing Subject, Style, and Action via Zero-Shot Personalizationcited by 2
  19. 2025Learning Joint ID-Textual Representation for ID-Preserving Image Synthesiscited by 2
  20. 2025Teleportraits: Training-Free People Insertion into Any Scenecited by 2
  21. 2024PersonaCraft: Personalized and Controllable Full-Body Multi-Human Scene Generation Using Occlusion-Aware 3D-Conditioned Diffusioncited by 1
  22. 2025Human Preference-Aligned Concept Customization Benchmark via Decomposed Evaluationcited by 1
  23. 2025Taming Identity Consistency and Prompt Diversity in Diffusion Models via Latent Concatenation and Masked Conditional Flow Matchingcited by 1
  24. 2025GaussianIP: Identity-Preserving Realistic 3D Human Generation via Human-Centric Diffusion Priorcited by 0
  25. 2025Retrieval Augmented Comic Image Generationcited by 0
  26. 2025A Training-Free Approach for Multi-ID Customization via Attention Adjustment and Spatial Controlcited by 0
  27. 2025Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalizationcited by 0
  28. 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvascited by 0
  29. 2026TimeWeaver: Age-Consistent Reference-Based Face Restoration with Identity Preservationcited by 0
  30. 2026Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generationcited by 0
  31. 2026PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generationcited by 0
  32. 2026MultiAnimate: A Unified Framework for Controllable Multi-Character Animationcited by 0
  33. 2026GroupVideo: Multi-Identity Customized Text-to-Video Generationcited by 0
  34. 2026WithEveryone: Unified Planning and Identity Grounding for Group Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward
  2. 2024MagicFace: Training-free Universal-Style Human Image Customized Synthesis
  3. 2026FaceSnap: Enhanced ID-fidelity Network for Tuning-free Portrait Customization
  4. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generation
  5. 2025Personalize Anything for Free with Diffusion Transformer
  6. 2025Reverse Personalization
  7. 2024RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance
  8. 2024MasterWeaver: Taming Editability and Face Identity for Personalized Text-to-Image Generation