mlsanityData to 7 Oct 2026

Fast Personalized Text-to-Image Syntheses With Attention Injection

arXiv 2403.11284 ↗First posted 17 Mar 2024cs.CVcited by 14

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Currently, personalized image generation methods mostly require considerable time to finetune and often overfit the concept resulting in generated images that are similar to custom concepts but difficult to edit by prompts. We propose an effective and fast approach that could balance the text-image consistency and identity consistency of the generated image and reference image. Our method can generate personalized images without any fine-tuning while maintaining the inherent text-to-image generation ability of diffusion models. Given a prompt and a reference image, we merge the custom concept into generated images by manipulating cross-attention and self-attention layers of the original diffusion model to generate personalized images that match the text description. Comprehensive experiments highlight the superiority of our method.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 19 of its 22 references that are in the map. First posted on arXiv 17 Mar 2024.

Builds on

19 of 22 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models5 Apr
  2. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  3. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  4. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug

Other references · 14

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  8. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  9. 2023Zero-shot Image-to-Image Translationcited by 503
  10. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  11. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  12. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
2 more
  1. 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
  2. 2023Face0: Instantaneously Conditioning a Text-to-Image Model on a Facecited by 77

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 10.

14 papers.

Sort:

By area:

  1. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  2. 2025Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Taskscited by 62
  3. 2025LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformercited by 43
  4. 2024ProcessPainter: Learn Painting Process from Sequence Datacited by 37
  5. 2024Stable-Hair: Real-World Hair Transfer via Diffusion Modelcited by 34
  6. 2025RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformerscited by 26
  7. 2025EasyText: Controllable Diffusion Transformer for Multilingual Text Renderingcited by 25
  8. 2025PhotoDoodle: Learning Artistic Image Editing from Few-Shot Pairwise Datacited by 23
  9. 2024IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generationcited by 21
  10. 2024DiffSim: Taming Diffusion Models for Evaluating Visual Similaritycited by 20
  11. 2025Stable-Hair v2: Real-World Hair Transfer via Multiple-View Diffusion Modelcited by 2
  12. 2025TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidancecited by 1
2 more
  1. 2025LLM-Enabled Style and Content Regularization for Personalized Text-to-Image Generationcited by 0
  2. 2025Animalbooth: multimodal feature enhancement for animal subject personalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Personalized Text-to-Image Generation with Auto-Regressive Models
  2. 2023CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization
  3. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
  4. 2024DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Models
  5. 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
  6. 2024Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learning
  7. 2024Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models
  8. 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model