mlsanityData to 13 Oct 2026

MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing

arXiv 2304.08465 ↗First posted 17 Apr 2023cs.CVcited by 725

Not on the map. Why?

Cited by 403 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the success in large-scale text-to-image generation and text-conditioned image editing, existing methods still struggle to produce consistent generation and editing results. For example, generation approaches usually fail to synthesize multiple images of the same objects/characters but with different views or poses. Meanwhile, existing editing methods either fail to achieve effective complex non-rigid editing while maintaining the overall textures and identity, or require time-consuming fine-tuning to capture the image-specific appearance. In this paper, we develop MasaCtrl, a tuning-free method to achieve consistent image generation and complex non-rigid image editing simultaneously. Specifically, MasaCtrl converts existing self-attention in diffusion models into mutual self-attention, so that it can query correlated local contents and textures from source images for consistency. To further alleviate the query confusion between foreground and background, we propose a mask-guided mutual self-attention strategy, where the mask can be easily extracted from the cross-attention maps. Extensive experiments show that the proposed MasaCtrl can produce impressive results in both consistent image generation and complex non-rigid real image editing.

Key facts. Cited by 725 papers in the map (data through 13 Oct 2026). Builds on 44 of its 47 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

44 of 47 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,106
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  7. 2020Denoising Diffusion Implicit Modelscited by 9,718
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,213
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,339
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,484
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,284
32 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,077
  2. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,751
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,294
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,237
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,619
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,421
  8. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,342
  9. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,289
  10. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,176
  11. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  12. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 959
  13. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 903
  14. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  15. 2016Generative Adversarial Text to Image Synthesiscited by 735
  16. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 701
  17. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 692
  18. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  19. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  20. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  21. 2023Zero-shot Image-to-Image Translationcited by 503
  22. 2022Blended Latent Diffusioncited by 432
  23. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  24. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  25. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  26. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 235
  27. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  28. 2019Controllable Text-to-Image Generationcited by 187
  29. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 161
  30. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  31. 2018Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Languagecited by 90
  32. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 72; 2024: 234; 2025: 265; 2026: 154.

725 papers.

Sort:

By area:

  1. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 463
  2. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 441
  3. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  4. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  5. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  6. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  7. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 241
  8. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 237
  9. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 232
  10. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  11. 2023Style Aligned Image Generation via Shared Attentioncited by 216
  12. 2024FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Modelscited by 215
38 more
  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2024Taming Rectified Flow for Inversion and Editingcited by 202
  3. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 202
  4. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 195
  5. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 194
  6. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Modelscited by 175
  7. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 174
  8. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 173
  9. 2023LEDITS++: Limitless Image Editing using Text-to-Image Modelscited by 147
  10. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Codecited by 145
  11. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  12. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 136
  13. 2024Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editingcited by 134
  14. 2024Recent Advances in 3D Gaussian Splattingcited by 130
  15. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  16. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  17. 2024Training-Free Consistent Text-to-Image Generationcited by 124
  18. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimizationcited by 123
  19. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Eraserscited by 120
  20. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  21. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 117
  22. 2024GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editingcited by 114
  23. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 113
  24. 2024One-Step Image Translation with Text-to-Image Modelscited by 113
  25. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 112
  26. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidancecited by 109
  27. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  28. 2023Inversion-Free Image Editing with Natural Languagecited by 106
  29. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 101
  30. 2024IMAGDressing-v1: Customizable Virtual Dressingcited by 100
  31. 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 100
  32. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generationcited by 93
  33. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 90
  34. 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 89
  35. 2024DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editingcited by 88
  36. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 88
  37. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  38. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesiscited by 84

50 most cited of 725 All 725 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection
  2. 2026LooseRoPE: Content-aware Attention Manipulation for Semantic Harmonization
  3. 2026Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength
  4. 2024Click2Mask: Local Editing with Dynamic Mask Generation
  5. 2025ReMix: Towards a Unified View of Consistent Character Generation and Editing
  6. 2026MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer
  7. 2025MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing
  8. 2026Editing Everything Everywhere All at Once