mlsanityData to 14 Oct 2026

CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts

arXiv 2503.24387 ↗First posted 31 Mar 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

While text-to-image generative models can synthesize diverse and faithful content, subject variation across multiple generations limits their application to long-form content generation. Existing approaches require time-consuming fine-tuning, reference images for all subjects, or access to previously generated content. We introduce Contrastive Concept Instantiation (CoCoIns), a framework that effectively synthesizes consistent subjects across multiple independent generations. The framework consists of a generative model and a mapping network that transforms input latent codes into pseudo-words associated with specific concept instances. Users can generate consistent subjects by reusing the same latent codes. To construct such associations, we propose a contrastive learning approach that trains the network to distinguish between different combinations of prompts and latent codes. Extensive evaluations on human faces with a single subject show that CoCoIns performs comparably to existing methods while maintaining greater flexibility. We also demonstrate the potential for extending CoCoIns to multiple subjects and other object categories.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 67 of its 76 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

67 of 76 references.

References · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2023Segment Anythingcited by 8,234
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
38 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  7. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  8. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  9. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  10. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  11. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  12. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  13. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  14. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  15. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  16. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  17. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  18. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  19. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  20. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  21. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 320
  22. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  23. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
  24. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  25. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
  26. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  27. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 212
  28. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  29. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  30. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  31. 2018StoryGAN: A Sequential Conditional GAN for Story Visualizationcited by 138
  32. 2024Training-Free Consistent Text-to-Image Generationcited by 125
  33. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 117
  34. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  35. 2023Face0: Instantaneously Conditioning a Text-to-Image Model on a Facecited by 78
  36. 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Modelscited by 77
  37. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
  38. 2022Make-A-Story: Visual Memory Conditioned Consistent Story Generationcited by 76

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.