mlsanityData to 7 Oct 2026

Delta-K: Boosting Multi-Instance Generation via Cross-Attention Augmentation

arXiv 2603.10210 ↗First posted 10 Mar 2026cs.CV cs.AIcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

While Diffusion Models excel in text-to-image synthesis, they frequently suffer from catastrophic concept omission when generating complex multi-instance scenes. Existing training-free methods attempt to resolve this by rescaling attention maps, which merely exacerbates unstructured noise without establishing coherent semantic representations. To address this, we propose Delta-K, a backbone-agnostic, plug-and-play inference framework that resolves omission by operating directly in the shared cross-attention Key space. Utilizing a lightweight Vision-Language Model (VLM) preview, we isolate a differential key (Delta K) capturing the pure semantic signature of missing concepts, and proactively inject it during the early semantic planning phase. Governed by a dynamically optimized scheduling mechanism, Delta-K grounds diffuse noise into stable structural anchors while naturally preserving existing concepts via the inherent orthogonality of Delta K. Extensive experiments validate its universal applicability, demonstrating that Delta-K significantly improves compositional alignment across both modern DiT and foundational U-Net architectures without requiring spatial masks, auxiliary training, or structural modifications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 62 of its 72 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

62 of 72 references.

Closest ancestors · 29

Same direction, newest first

  1. 2026TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models12 Jan
  2. 2026FreeText: Training-Free Text Rendering in Diffusion Transformers via Attention Localization and Spectral Glyph Injection2 Jan
  3. 2025Detail++: Training-Free Detail Enhancer for T2I Diffusion Models23 Jul
  4. 2025Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion26 May
  5. 2025MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation5 May
  6. 2025Compass Control: Multi Object Orientation Control for Text-to-Image Generation9 Apr
  7. 2024Self-Cross Diffusion Guidance for Text-to-Image Synthesis of Similar Subjects28 Nov
  8. 2024Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance29 Oct
  9. 2024Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models17 Jun
  10. 2024Make It Count: Text-to-Image Generation with an Accurate Number of Objects14 Jun
  11. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization6 Apr
  12. 2024CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching4 Apr
17 more
  1. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation27 Feb
  2. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs22 Jan
  3. 2023CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models11 Dec
  4. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  5. 2023Improving Compositional Text-to-image Generation with Large Vision-Language Models10 Oct
  6. 2023Dense Text-to-Image Generation with Attention Modulation24 Aug
  7. 2023Divide & Bind Your Attention for Improved Generative Semantic Nursing20 Jul
  8. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion20 Jul
  9. 2023A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesis26 Jun
  10. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment15 Jun
  11. 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
  12. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  13. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  14. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  15. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  16. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  17. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 33

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2025Qwen3-VL Technical Reportcited by 2,534
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
21 more
  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  2. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  3. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  4. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  5. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  6. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  7. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  8. 2025Seedream 3.0 Technical Reportcited by 146
  9. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  10. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  11. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generationcited by 73
  12. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficultycited by 60
  13. 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Modelscited by 59
  14. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
  15. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 36
  16. 2024Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesiscited by 35
  17. 2025LD-Scene: LLM-Guided Diffusion for Controllable Generation of Adversarial Safety-Critical Driving Scenarioscited by 9
  18. 2025Compositional Diffusion with Guided Search for Long-Horizon Planningcited by 8
  19. 2024Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generationcited by 5
  20. 2024Concept Conductor: Orchestrating Multiple Personalized Concepts in Text-to-Image Synthesiscited by 4
  21. 2025VLM-Guided Adaptive Negative Prompting for Creative Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025UniFusion: Vision-Language Model as Unified Encoder in Image Generation
  2. 2024Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis
  3. 2025UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception
  4. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
  5. 2025ISAC: Training-Free Instance-to-Semantic Attention Control for Multi-Instance Generation
  6. 2025MammothModa2: A Unified AR-Diffusion Framework for Multimodal Understanding and Generation
  7. 2025I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models
  8. 2026Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models