mlsanityData to 7 Oct 2026

AID: Attention Interpolation of Text-to-Image Diffusion

arXiv 2403.17924 ↗First posted 26 Mar 2024cs.CV cs.AIcited by 24

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Conditional diffusion models can create unseen images in various settings, aiding image interpolation. Interpolation in latent spaces is well-studied, but interpolation with specific conditions like text or poses is less understood. Simple approaches, such as linear interpolation in the space of conditions, often result in images that lack consistency, smoothness, and fidelity. To that end, we introduce a novel training-free technique named Attention Interpolation via Diffusion (AID). Our key contributions include 1) proposing an inner/outer interpolated attention layer; 2) fusing the interpolated attention with self-attention to boost fidelity; and 3) applying beta distribution to selection to increase smoothness. We also present a variant, Prompt-guided Attention Interpolation via Diffusion (PAID), that considers interpolation as a condition-dependent generative process. This method enables the creation of new images with greater consistency, smoothness, and efficiency, and offers control over the exact path of interpolation. Our approach demonstrates effectiveness for conceptual and spatial interpolation. Code and demo are available at https://github.com/QY-H00/attention-interpolation-diffusion.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 47 of its 60 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

47 of 60 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Font Style Interpolation with Diffusion Models22 Feb
  2. 2023DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing12 Dec
  3. 2023IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models12 Nov
  4. 2023Interpolating between Images with Diffusion Models24 Jul
  5. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment15 Jun
  6. 2023Norm-guided latent space exploration for text-to-image generation14 Jun
  7. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models23 May
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  10. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2022Scalable Diffusion Models with Transformerscited by 5,931
  12. 2022Generative Adversarial Networkscited by 5,603
23 more
  1. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  8. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  9. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  10. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  11. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  12. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
  13. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 393
  14. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  15. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 232
  16. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  17. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  18. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 151
  19. 2020Interpreting Super-Resolution Networks with Local Attribution Mapscited by 136
  20. 2022Understanding DDPM Latent Codes Through Optimal Transportcited by 44
  21. 2020Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Modelscited by 26
  22. 2023Pair then Relation: Pair-Net for Panoptic Scene Graph Generationcited by 17
  23. 2023Real-World Denoising via Diffusion Modelcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 18; 2026: 5.

24 papers.

Sort:

By area:

  1. 2025FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Modelcited by 17
  2. 2025Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Spacecited by 14
  3. 2025Probability Density Geodesics in Image Diffusion Latent Spacecited by 9
  4. 2025FreeBlend: Advancing Concept Blending with Staged Feedback-Driven Interpolation Diffusioncited by 8
  5. 20253D Stylization via Large Reconstruction Modelcited by 8
  6. 2025Versatile Transition Generation with Image-to-Video Diffusioncited by 7
  7. 2025Textured 3D Regenerative Morphing with 3D Diffusion Priorcited by 6
  8. 2025REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularizationcited by 6
  9. 2025Cora: Correspondence-aware image editing using few step diffusioncited by 5
  10. 2025AttriCtrl: Fine-Grained Control of Aesthetic Attribute Intensity in Diffusion Modelscited by 5
  11. 2025Attention in Diffusion Model: A Surveycited by 4
  12. 2026MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphingcited by 4
12 more
  1. 2026Interp3D: Correspondence-aware Interpolation for Generative Textured 3D Morphingcited by 4
  2. 2025Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Modelscited by 3
  3. 2025Vibe Spaces for Creatively Connecting and Expressing Visual Conceptscited by 3
  4. 2025SemMorph3D: Unsupervised Semantic-Aware 3D Morphing via Mesh-Guided Gaussianscited by 1
  5. 2025Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Modelscited by 1
  6. 2025Which Way from B to A: The role of embedding geometry in image interpolation for Stable Diffusioncited by 1
  7. 2026VPG: Visual Prefix Guidance for Autoregressive Image and Video Generationcited by 1
  8. 2024From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Taskcited by 0
  9. 2025"I Know It When I See It": Mood Spaces for Connecting and Expressing Visual Conceptscited by 0
  10. 2025Blending Concepts with Text-to-Image Diffusion Modelscited by 0
  11. 2026PureCC: Pure Learning for Text-to-Image Concept Customizationcited by 0
  12. 2026Attention Frequency Modulation: Training-Free Spectral Modulation of Diffusion Cross-Attentioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models
  2. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
  3. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis
  4. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization
  5. 2024Enhancing Image Layout Control with Loss-Guided Diffusion Models
  6. 2024CoCoNO: Attention Contrast-and-Complete for Initial Noise Optimization in Text-to-Image Synthesis
  7. 2024Enhancing Conditional Image Generation with Explainable Latent Space Manipulation
  8. 2026AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis