# Divide & Bind Your Attention for Improved Generative Semantic Nursing

arXiv: https://arxiv.org/abs/2307.10864 · HTML: https://mlsanity.com/paper/2307.10864/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [What the DAAM: Interpreting Stable Diffusion Using Cross Attention](https://mlsanity.com/paper/2210.04885/) (2022)
- [Teaching CLIP to Count to Ten](https://mlsanity.com/paper/2302.12066/) (2023)
- [mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections](https://mlsanity.com/paper/2205.12005/) (2022)
- [LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation](https://mlsanity.com/paper/2305.11116/) (2023)
- [A very preliminary analysis of DALL-E 2](https://mlsanity.com/paper/2204.13807/) (2022)
- [MagicMix: Semantic Mixing with Diffusion Models](https://mlsanity.com/paper/2210.16056/) (2022)
- [LAVIS: A Library for Language-Vision Intelligence](https://mlsanity.com/paper/2209.09019/) (2022)
- [In-Context Learning for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2203.08568/) (2022)

## Built on by (most cited first, 30 of 81)

- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Cross-Image Attention for Zero-Shot Appearance Transfer](https://mlsanity.com/paper/2311.03335/) (2023)
- [InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization](https://mlsanity.com/paper/2404.04650/) (2024)
- [GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs](https://mlsanity.com/paper/2312.00093/) (2023)
- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching](https://mlsanity.com/paper/2404.03653/) (2024)
- [CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models](https://mlsanity.com/paper/2312.06059/) (2023)
- [SwiftBrush v2: Make Your One-step Diffusion Model Better Than Its Teacher](https://mlsanity.com/paper/2408.14176/) (2024)
- [RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization](https://mlsanity.com/paper/2403.00483/) (2024)
- [MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesis](https://mlsanity.com/paper/2407.02329/) (2024)
- [Alignment of Diffusion Models: Fundamentals, Challenges, and Future](https://mlsanity.com/paper/2409.07253/) (2024)
- [Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis](https://mlsanity.com/paper/2411.07132/) (2024)
- [Quantum circuit synthesis with diffusion models](https://mlsanity.com/paper/2311.02041/) (2023)
- [R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation](https://mlsanity.com/paper/2310.08872/) (2023)
- [MC2: Multi-concept Guidance for Customized Multi-concept Generation](https://mlsanity.com/paper/2404.05268/) (2024)
- [SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance](https://mlsanity.com/paper/2405.15321/) (2024)
- [Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers](https://mlsanity.com/paper/2506.07986/) (2025)
- [Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models](https://mlsanity.com/paper/2312.06712/) (2023)
- [The Crystal Ball Hypothesis in diffusion models: Anticipating object positions from initial noise](https://mlsanity.com/paper/2406.01970/) (2024)
- [Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive](https://mlsanity.com/paper/2401.08815/) (2024)
- [A Survey on Quality Metrics for Text-to-Image Generation](https://mlsanity.com/paper/2403.11821/) (2024)
- [Non-confusing Generation of Customized Concepts in Diffusion Models](https://mlsanity.com/paper/2405.06914/) (2024)
- [The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generation](https://mlsanity.com/paper/2412.05101/) (2024)
- [Towards Transformer-Based Aligned Generation with Self-Coherence Guidance](https://mlsanity.com/paper/2503.17675/) (2025)
- [FlexEdit: Flexible and Controllable Diffusion-based Object-centric Image Editing](https://mlsanity.com/paper/2403.18605/) (2024)
- [Progressive Compositionality in Text-to-Image Generative Models](https://mlsanity.com/paper/2410.16719/) (2024)
- [VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis](https://mlsanity.com/paper/2403.13501/) (2024)
- [Training-free Composite Scene Generation for Layout-to-Image Synthesis](https://mlsanity.com/paper/2407.13609/) (2024)
- [Contrastive Test-Time Composition of Multiple LoRA Models for Image Generation](https://mlsanity.com/paper/2403.19776/) (2024)
- [AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation](https://mlsanity.com/paper/2406.05000/) (2024)

## Abstract (arXiv)

Emerging large-scale text-to-image generative models, e.g., Stable Diffusion (SD), have exhibited overwhelming results with high fidelity. Despite the magnificent progress, current state-of-the-art models still struggle to generate images fully adhering to the input prompt. Prior work, Attend & Excite, has introduced the concept of Generative Semantic Nursing (GSN), aiming to optimize cross-attention during inference time to better incorporate the semantics. It demonstrates promising results in generating simple prompts, e.g., "a cat and a dog". However, its efficacy declines when dealing with more complex prompts, and it does not explicitly address the problem of improper attribute binding. To address the challenges posed by complex prompts or scenarios involving multiple entities and to achieve improved attribute binding, we propose Divide & Bind. We introduce two novel loss objectives for GSN: a novel attendance loss and a binding loss. Our approach stands out in its ability to faithfully synthesize desired objects with improved attribute alignment from complex prompts and exhibits superior performance across multiple evaluation benchmarks.