mlsanityData to 7 Oct 2026

InstantID: Zero-shot Identity-Preserving Generation in Seconds

arXiv 2401.07519 ↗First posted 15 Jan 2024cs.CV cs.AIcited by 431

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at https://github.com/InstantID/InstantID.

Key facts. Cited by 431 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 25 of its 28 references that are in the map. First posted on arXiv 15 Jan 2024.

Builds on

25 of 28 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  2. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  9. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  10. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  11. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  12. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
4 more
  1. 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 193
  2. 2023Face0: Instantaneously Conditioning a Text-to-Image Model on a Facecited by 77
  3. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  4. 2023Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approachcited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 160; 2025: 187; 2026: 84.

431 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2025VACE: All-in-One Video Creation and Editingcited by 467
  3. 2024OmniGen: Unified Image Generationcited by 388
  4. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  5. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
  6. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  7. 2024InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generationcited by 166
  8. 2024In-Context LoRA for Diffusion Transformerscited by 155
  9. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  10. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
  11. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
  12. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 143
38 more
  1. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  2. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  3. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  4. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 122
  5. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  6. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 111
  7. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  8. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
  9. 2024CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Modelscited by 99
  10. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  11. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 85
  12. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  13. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  14. 2024CSGO: Content-Style Composition in Text-to-Image Generationcited by 76
  15. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  16. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  17. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  18. 2024Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Controlcited by 66
  19. 2025Multi-subject Open-set Personalization in Video Generationcited by 65
  20. 2024Common Sense Reasoning for Deepfake Detectioncited by 64
  21. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Modelscited by 64
  22. 2025Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Taskscited by 62
  23. 2025Insert Anything: Image Insertion via In-Context Editing in DiTcited by 61
  24. 2024StyleShot: A Snapshot on Any Stylecited by 60
  25. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  26. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 58
  27. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 57
  28. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  29. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
  30. 2024VideoTetris: Towards Compositional Text-to-Video Generationcited by 51
  31. 2024InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generationcited by 51
  32. 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 49
  33. 2025Is Artificial Intelligence Generated Image Detection a Solved Problem?cited by 49
  34. 2024AnyText2: Visual Text Generation and Editing With Customizable Attributescited by 48
  35. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 47
  36. 2024RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Controlcited by 46
  37. 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 46
  38. 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generationcited by 46

50 most cited of 431 All 431 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023When StyleGAN Meets Stable Diffusion: a mathscrW_+ Adapter for Personalized Image Generation
  2. 2024Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm
  3. 2024InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention
  4. 2026Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models
  5. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization
  6. 2026Zero-Shot Personalization of Objects via Textual Inversion
  7. 2024LCM-Lookahead for Encoder-based Text-to-Image Personalization
  8. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models