CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts
Not on the map. Why?
Abstract (arXiv)
While text-to-image generative models can synthesize diverse and faithful content, subject variation across multiple generations limits their application to long-form content generation. Existing approaches require time-consuming fine-tuning, reference images for all subjects, or access to previously generated content. We introduce Contrastive Concept Instantiation (CoCoIns), a framework that effectively synthesizes consistent subjects across multiple independent generations. The framework consists of a generative model and a mapping network that transforms input latent codes into pseudo-words associated with specific concept instances. Users can generate consistent subjects by reusing the same latent codes. To construct such associations, we propose a contrastive learning approach that trains the network to distinguish between different combinations of prompts and latent codes. Extensive evaluations on human faces with a single subject show that CoCoIns performs comparably to existing methods while maintaining greater flexibility. We also demonstrate the potential for extending CoCoIns to multiple subjects and other object categories.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 67 of its 76 references that are in the map. First posted on arXiv 31 Mar 2025.
Builds on
67 of 76 references.
References · 67
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2023Segment Anythingcited by 8,234
- 2023Visual Instruction Tuningcited by 7,869
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2014Deep Learning Face Attributes in the Wildcited by 3,693
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
38 more
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
- 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
- 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
- 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
- 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
- 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
- 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
- 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 320
- 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
- 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
- 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
- 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
- 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 212
- 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
- 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
- 2018StoryGAN: A Sequential Conditional GAN for Story Visualizationcited by 138
- 2024Training-Free Consistent Text-to-Image Generationcited by 125
- 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 117
- 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
- 2023Face0: Instantaneously Conditioning a Text-to-Image Model on a Facecited by 78
- 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Modelscited by 77
- 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
- 2022Make-A-Story: Visual Memory Conditioned Consistent Story Generationcited by 76
50 most cited of 67 All 67 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.