mlsanityData to 7 Oct 2026

ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation

arXiv 2302.13848 ↗First posted 27 Feb 2023cs.CVcited by 414

Not on the map. Why?

Cited by 207 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In addition to the unprecedented ability in imaginary creation, large text-to-image models are expected to take customized concepts in image generation. Existing works generally learn such concepts in an optimization-based manner, yet bringing excessive computation or memory burden. In this paper, we instead propose a learning-based encoder, which consists of a global and a local mapping networks for fast and accurate customized text-to-image generation. In specific, the global mapping network projects the hierarchical features of a given image into multiple new words in the textual word embedding space, i.e., one primary word for well-editable concept and other auxiliary words to exclude irrelevant disturbances (e.g., background). In the meantime, a local mapping network injects the encoded patch features into cross attention layers to provide omitted details, without sacrificing the editability of primary concepts. We compare our method with existing optimization-based approaches on a variety of user-defined concepts, and demonstrate that our method enables high-fidelity inversion and more robust editability with a significantly faster encoding process. Our code is publicly available at https://github.com/csyxwei/ELITE.

Key facts. Cited by 414 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 27 Feb 2023.

Builds on

36 of 45 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
24 more
  1. 2018Demystifying MMD GANscited by 1,268
  2. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  3. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  5. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  6. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  7. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  8. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  9. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  10. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  11. 2021GAN Inversion: A Surveycited by 366
  12. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  13. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  14. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  15. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  16. 2019Controllable Text-to-Image Generationcited by 187
  17. 2017Optimizing the Latent Space of Generative Networkscited by 157
  18. 2016Inverting The Generator Of A Generative Adversarial Networkcited by 140
  19. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  20. 2022Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Modelscited by 69
  21. 2022Spatially-Adaptive Multilayer Selection for GAN Inversion and Editingcited by 39
  22. 2022Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentationcited by 28
  23. 2022PP-Matting: High-Accuracy Natural Image Mattingcited by 23
  24. 2022ImaginaryNet: Learning Object Detectors without Real Images and Annotationscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 84; 2024: 180; 2025: 111; 2026: 39.

414 papers.

Sort:

By area:

  1. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  2. 2024OmniGen: Unified Image Generationcited by 388
  3. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  4. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  5. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  6. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  7. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  8. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  9. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 284
  10. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  11. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  12. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
38 more
  1. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  2. 2023Style Aligned Image Generation via Shared Attentioncited by 214
  3. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  4. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  5. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  6. 2023Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Modelscited by 172
  7. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 172
  8. 2023AnyText: Multilingual Visual Text Generation And Editingcited by 165
  9. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  10. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
  11. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  12. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 143
  13. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  14. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  15. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  16. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  17. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 124
  18. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  19. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  20. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  21. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  22. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
  23. 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 92
  24. 2023Animate-A-Story: Storytelling with Retrieval-Augmented Video Generationcited by 89
  25. 2024DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editingcited by 88
  26. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  27. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Modelscited by 86
  28. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  29. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  30. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
  31. 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
  32. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  33. 2023PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion Modelscited by 70
  34. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  35. 2025Multi-subject Open-set Personalization in Video Generationcited by 65
  36. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Modelscited by 64
  37. 2023Inserting Anybody in Diffusion Models via Celeb Basiscited by 63
  38. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 62

50 most cited of 414 All 414 (JSON)