mlsanityData to 7 Oct 2026

The Chosen One: Consistent Characters in Text-to-Image Diffusion Models

arXiv 2311.10093 ↗First posted 16 Nov 2023cs.CV cs.GR cs.LGcited by 62

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Recent advances in text-to-image generation models have unlocked vast potential for visual creativity. However, the users that use these models struggle with the generation of consistent characters, a crucial aspect for numerous real-world applications such as story visualization, game development, asset design, advertising, and more. Current methods typically rely on multiple pre-existing images of the target character or involve labor-intensive manual processes. In this work, we propose a fully automated solution for consistent character generation, with the sole input being a text prompt. We introduce an iterative procedure that, at each stage, identifies a coherent set of images sharing a similar identity and extracts a more consistent identity from this set. Our quantitative analysis demonstrates that our method strikes a better balance between prompt alignment and identity consistency compared to the baseline methods, and these findings are reinforced by a user study. To conclude, we showcase several practical applications of our approach.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 80 of its 99 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

80 of 99 references.

Closest ancestors · 24

Same direction, newest first

  1. 2023DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models13 Sep
  2. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  3. 2023Zero-shot spatial layout conditioning for text-to-image diffusion models23 Jun
  4. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models5 Apr
  5. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Models20 Mar
  6. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  7. 2023Text-to-image Diffusion Models in Generative AI: A Survey14 Mar
  8. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models23 Feb
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  11. 2023Zero-shot Generation of Coherent Storybook from Plain Text Story using Diffusion Models8 Feb
  12. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
12 more
  1. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  2. 2022SpaText: Spatio-Textual Representation for Controllable Image Generation25 Nov
  3. 2022Sketch-Guided Text-to-Image Diffusion Models24 Nov
  4. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  6. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  8. 2022Blended Latent Diffusion6 Jun
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  12. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 56

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
38 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2022Human Motion Diffusion Modelcited by 1,062
  6. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  7. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  8. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  9. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  10. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  11. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  12. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  13. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  14. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  15. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  16. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  17. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  18. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  19. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  20. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  21. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  22. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  23. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  24. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 189
  25. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  26. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  27. 2018StoryGAN: A Sequential Conditional GAN for Story Visualizationcited by 137
  28. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  29. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  30. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  31. 2023Delta Denoising Scorecited by 120
  32. 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 114
  33. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  34. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  35. 2023Face0: Instantaneously Conditioning a Text-to-Image Model on a Facecited by 77
  36. 2022Make-A-Story: Visual Memory Conditioned Consistent Story Generationcited by 76
  37. 2022StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuationcited by 74
  38. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 71

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 23; 2025: 24; 2026: 13.

62 papers.

Sort:

By area:

  1. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
  2. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  3. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  4. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  5. 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 70
  6. 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 49
  7. 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
  8. 2024DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusioncited by 46
  9. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  10. 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Modelscited by 24
  11. 2025Personalize Anything for Free with Diffusion Transformercited by 22
  12. 2025Nested Attention: Semantic-aware Attention Values for Concept Personalizationcited by 21
38 more
  1. 2023Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Modelscited by 19
  2. 2024AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generationcited by 19
  3. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  4. 2025Story2Board: A Training-Free Approach for Expressive Storyboard Generationcited by 16
  5. 2024OneActor: Consistent Character Generation via Cluster-Conditioned Guidancecited by 15
  6. 2024Unbounded: A Generative Infinite Game of Character Life Simulationcited by 14
  7. 2024Compositional Image Decomposition with Diffusion Modelscited by 12
  8. 2024Story-Iter: A Training-free Iterative Paradigm for Long Story Visualizationcited by 12
  9. 2026Group Editing: Edit Multiple Images in One Gocited by 11
  10. 2023CogCartoon: Towards Practical Story Visualizationcited by 8
  11. 2024ORACLE: Leveraging Mutual Information for Consistent Character Generation with LoRAs in Diffusion Modelscited by 7
  12. 2025SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistencycited by 7
  13. 2025Geometry-Aware Diffusion Models for Multiview Scene Inpaintingcited by 6
  14. 2025Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generationcited by 6
  15. 2024CAT: Contrastive Adapter Training for Personalized Image Generationcited by 5
  16. 2024RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidancecited by 4
  17. 2025Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytellingcited by 4
  18. 2025C3Editor: Achieving Controllable Consistency in 2D Model for 3D Editingcited by 4
  19. 2024Generative Photomontagecited by 3
  20. 2024Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attentioncited by 3
  21. 2024Motion by Queries: Identity-Motion Trade-offs in Text-to-Video Generationcited by 3
  22. 2025ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Modelscited by 3
  23. 2025In-Context Sync-LoRA for Portrait Video Editingcited by 3
  24. 2024EZIGen: Enhancing zero-shot personalized image generation with precise subject encoding and decoupled guidancecited by 2
  25. 2024Adaptive Multi-Modal Control of Digital Human Hand Synthesis Using a Region-Aware Cycle Losscited by 2
  26. 2025Learning Zero-Shot Subject-Driven Video Generation Using 1% Computecited by 2
  27. 2025Consistent Story Generation: Unlocking the Potential of Zigzag Samplingcited by 2
  28. 2025CharCom: Composable Identity Control for Multi-Character Story Illustrationcited by 2
  29. 2024BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversioncited by 1
  30. 2025CoCoIns: Consistent Subject Generation via Contrastive Instantiated Conceptscited by 1
  31. 2025ConsiStyle: Style Diversity in Training-Free Consistent T2I Generationcited by 1
  32. 2025CoDi: Subject-Consistent and Pose-Diverse Text-to-Image Generationcited by 1
  33. 2025StorySync: Training-Free Subject Consistency in Text-to-Image Generation via Region Harmonizationcited by 1
  34. 2025Plot'n Polish: Zero-shot Story Visualization and Disentangled Editing with Text-to-Image Diffusion Modelscited by 1
  35. 2025Consistent text-to-image generation via scene de-contextualizationcited by 1
  36. 2026ReCap: Lightweight Referential Grounding for Coherent Story Visualizationcited by 1
  37. 2026Text-to-Image Generative AI for Modeling and Simulation: Methods, Opportunities, and Applicationscited by 1
  38. 2026FreeStory: Training-Free Character Consistency for Free-Form Visual Storytellingcited by 1

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt
  2. 2024Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models
  3. 2024Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Models
  4. 2023ObjectComposer: Consistent Generation of Multiple Objects Without Fine-tuning
  5. 2024All Seeds Are Not Equal: Enhancing Compositional Text-to-Image Generation with Reliable Random Seeds
  6. 2024Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models
  7. 2024ARTIST: Improving the Generation of Text-rich Images with Disentangled Diffusion Models and Large Language Models
  8. 2025DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models