mlsanityData to 7 Oct 2026

Prompt-to-Prompt Image Editing with Cross Attention Control

arXiv 2208.01626 ↗First posted 2 Aug 2022cs.CV cs.CL cs.GRcited by 2,409

Not on the map. Why?

Cited by 1,373 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent large-scale text-driven synthesis models have attracted much attention thanks to their remarkable capabilities of generating highly diverse images that follow given text prompts. Such text-based synthesis methods are particularly appealing to humans who are used to verbally describe their intent. Therefore, it is only natural to extend the text-driven image synthesis to text-driven image editing. Editing is challenging for these generative models, since an innate property of an editing technique is to preserve most of the original image, while in the text-based models, even a small modification of the text prompt often leads to a completely different outcome. State-of-the-art methods mitigate this by requiring the users to provide a spatial mask to localize the edit, hence, ignoring the original structure and content within the masked region. In this paper, we pursue an intuitive prompt-to-prompt editing framework, where the edits are controlled by text only. To this end, we analyze a text-conditioned model in depth and observe that the cross-attention layers are the key to controlling the relation between the spatial layout of the image to each word in the prompt. With this observation, we present several applications which monitor the image synthesis by editing the textual prompt only. This includes localized editing by replacing a word, global editing by adding a specification, and even delicately controlling the extent to which a word is reflected in the image. We present our results over diverse images and prompts, demonstrating high-quality synthesis and fidelity to the edited prompts.

Key facts. Cited by 2,409 papers in the map (data through 7 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 2 Aug 2022.

Builds on

54 of 68 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  10. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  11. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  12. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  13. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  14. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  15. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  16. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  17. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  18. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  19. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  20. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  21. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  22. 2022Blended Latent Diffusioncited by 430
  23. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  24. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  25. 2021GAN Inversion: A Surveycited by 366
  26. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  27. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  28. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  29. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  30. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  31. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  32. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  33. 2019Controllable Text-to-Image Generationcited by 187
  34. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  35. 2017Fader Networks: Manipulating Images by Sliding Attributescited by 118
  36. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  37. 2021Paint by Wordcited by 107
  38. 2019Semantic Object Accuracy for Generative Text-to-Image Synthesiscited by 89

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 43; 2023: 526; 2024: 721; 2025: 650; 2026: 469.

2,409 papers.

Sort:

By area:

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  3. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  6. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  8. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  11. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  12. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
38 more
  1. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  2. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  3. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  4. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  5. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  6. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  7. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  8. 2023Zero-shot Image-to-Image Translationcited by 503
  9. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  10. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  11. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  12. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 486
  13. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  14. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  15. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  16. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  17. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  18. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  19. 2023Emergent Correspondence from Image Diffusioncited by 437
  20. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  21. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  22. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  23. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  24. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  25. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  26. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  27. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  28. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  29. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  30. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  31. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  32. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  33. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  34. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  35. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  36. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  37. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  38. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolutioncited by 311

50 most cited of 2,409 All 2,409 (JSON)