mlsanityData to 14 Oct 2026

Face0: Instantaneously Conditioning a Text-to-Image Model on a Face

arXiv 2306.06638 ↗First posted 11 Jun 2023cs.CV cs.AI cs.GRcited by 78

Not on the map. Why?

Cited by 31 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Face0, a novel way to instantaneously condition a text-to-image generation model on a face, in sample time, without any optimization procedures such as fine-tuning or inversions. We augment a dataset of annotated images with embeddings of the included faces and train an image generation model, on the augmented dataset. Once trained, our system is practically identical at inference time to the underlying base model, and is therefore able to generate images, given a user-supplied face image and a prompt, in just a couple of seconds. Our method achieves pleasing results, is remarkably simple, extremely fast, and equips the underlying model with new capabilities, like controlling the generated images both via text or via direct manipulation of the input face embeddings. In addition, when using a fixed random vector instead of a face embedding from a user supplied image, our method essentially solves the problem of consistent character generation across images. Finally, while requiring further research, we hope that our method, which decouples the model's textual biases from its biases on faces, might be a step towards some mitigation of biases in future text-to-image models.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Builds on 29 of its 34 references that are in the map. First posted on arXiv 11 Jun 2023.

Builds on

29 of 34 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  9. 2022Classifier-Free Diffusion Guidancecited by 5,847
  10. 2022Generative Adversarial Networkscited by 5,627
  11. 2017Neural Discrete Representation Learningcited by 4,712
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,288
17 more
  1. 2014Going Deeper with Convolutionscited by 4,241
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  6. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  8. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  9. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  10. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  11. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  12. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  13. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  14. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 191
  15. 2021One Shot Face Swapping on Megapixelscited by 105
  16. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  17. 2022MyStyle: A Personalized Generative Priorcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 33; 2025: 28; 2026: 6.

78 papers.

Sort:

By area:

  1. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  2. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  3. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 144
  4. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  5. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  6. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  7. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 124
  8. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  9. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  10. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
  11. 2025Multi-subject Open-set Personalization in Video Generationcited by 66
  12. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 63
38 more
  1. 2023FaceStudio: Put Your Face Everywhere in Secondscited by 53
  2. 2024ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preservingcited by 43
  3. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  4. 2024PALP: Prompt Aligned Personalization of Text-to-Image Modelscited by 35
  5. 2024IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Modelscited by 32
  6. 2024InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generationcited by 30
  7. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  8. 2024PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradationcited by 29
  9. 2024Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Modelscited by 23
  10. 2024Group Diffusion Transformers are Unsupervised Multitask Learnerscited by 22
  11. 2024IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generationcited by 22
  12. 2025Nested Attention: Semantic-aware Attention Values for Concept Personalizationcited by 21
  13. 2024PreciseControl: Enhancing Text-To-Image Diffusion Models with Fine-Grained Attribute Controlcited by 20
  14. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  15. 2024ID3: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognitioncited by 18
  16. 2025MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customizationcited by 17
  17. 2025WithAnyone: Towards Controllable and ID Consistent Image Generationcited by 17
  18. 2025Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidancecited by 16
  19. 2024Fast Personalized Text-to-Image Syntheses With Attention Injectioncited by 15
  20. 2024Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectorscited by 15
  21. 2023A Generalist FaceX via Learning Unified Facial Representationcited by 14
  22. 2024Customizing Text-to-Image Diffusion with Object Viewpoint Controlcited by 13
  23. 2024DiffLoRA: Generating Personalized Low-Rank Adaptation Weights with Diffusioncited by 12
  24. 2025Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusioncited by 12
  25. 2023SingleInsert: Inserting New Concepts from a Single Image into Text-to-Image Models for Flexible Editingcited by 10
  26. 2024AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generationcited by 10
  27. 2025DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editabilitycited by 10
  28. 2024Multi-modal Generative AI: Multi-modal LLMs, Diffusions, and the Unificationcited by 8
  29. 2023Viewpoint Textual Inversion: Discovering Scene Representations and 3D View Control in 2D Diffusion Modelscited by 7
  30. 2025Visual Persona: Foundation Model for Full-Body Human Customizationcited by 7
  31. 2023When StyleGAN Meets Stable Diffusion: a mathscrW_+ Adapter for Personalized Image Generationcited by 6
  32. 2024Improving Subject-Driven Image Synthesis with Subject-Agnostic Guidancecited by 6
  33. 2025Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generationcited by 6
  34. 2025InstaFace: Identity-Preserving Facial Editing with Single Image Inferencecited by 5
  35. 2024ControlFace: Harnessing Facial Parametric Control for Face Riggingcited by 4
  36. 2025FaceID-6M: A Large-Scale, Open-Source FaceID Customization Datasetcited by 4
  37. 2025Diffusion-based Facial Aesthetics Enhancement with 3D Structure Guidancecited by 4
  38. 2025FaceCrafter: Identity-Conditional Diffusion with Disentangled Control over Facial Pose, Expression, and Emotioncited by 4

50 most cited of 78 All 78 (JSON)