Training-Free Object-Background Compositional T2I via Dynamic Spatial Guidance and Multi-Path Pruning
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Existing text-to-image diffusion models, while excelling at subject synthesis, exhibit a persistent foreground bias that treats the background as a passive and under-optimized byproduct. This imbalance compromises global scene coherence and constrains compositional control. To address the limitation, we propose a training-free framework that restructures diffusion sampling to explicitly account for foreground-background interactions. Our approach consists of two key components. First, Dynamic Spatial Guidance introduces a soft, time step dependent gating mechanism that modulates foreground and background attention during the diffusion process, enabling spatially balanced generation. Second, Multi-Path Pruning performs multi-path latent exploration and dynamically filters candidate trajectories using both internal attention statistics and external semantic alignment signals, retaining trajectories that better satisfy object-background constraints. We further develop a benchmark specifically designed to evaluate object-background compositionality. Extensive evaluations across multiple diffusion backbones demonstrate consistent improvements in background coherence and object-background compositional alignment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 51 of its 61 references that are in the map. First posted on arXiv 10 Apr 2026.
Builds on
51 of 61 references.
Closest ancestors · 22
Same direction, newest first
- 2025MaskAttn-SDXL: Controllable Region-Level Text-To-Image Generation18 Sep
- 2025Be Decisive: Noise-Induced Layouts for Multi-Subject Generation27 May
- 2025MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation5 May
- 2025Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesis28 Mar
- 2025Towards Transformer-Based Aligned Generation with Self-Coherence Guidance22 Mar
- 2024LLM4GEN: Leveraging Semantic Representation of LLMs for Text-to-Image Generation30 Jun
- 2024The Crystal Ball Hypothesis in diffusion models: Anticipating object positions from initial noise4 Jun
- 2024Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Models10 Apr
- 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization6 Apr
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment8 Mar
- 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation27 Feb
- 2024RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models20 Feb
10 more
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs22 Jan
- 2023CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models11 Dec
- 2023Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models10 Dec
- 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervision6 Dec
- 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following28 Nov
- 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models23 May
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar
- 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Models20 Mar
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
Other references · 29
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
17 more
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
- 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
- 2023Style Aligned Image Generation via Shared Attentioncited by 216
- 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
- 2025Deep Think with Confidencecited by 205
- 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
- 2015Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labelscited by 64
- 2023Relightful Harmonization: Lighting-aware Portrait Background Replacementcited by 57
- 2022DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Modelscited by 51
- 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
- 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33
- 2025LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integrationcited by 19
- 2025Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoningcited by 18
- 2025CER: Confidence Enhanced Reasoning in LLMscited by 17
- 2024SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generationcited by 15
- 2015Reconciling saliency and object center-bias hypotheses in explaining free-viewing fixationscited by 6
- 2025Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Abilitycited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis
- 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
- 2025Debiasing Diffusion Priors via 3D Attention for Consistent Gaussian Splatting
- 2025FreeControl: Efficient, Training-Free Structural Control via One-Step Attention Extraction
- 2026Focal Guidance: Unlocking Controllability from Semantic-Weak Layers in Video Diffusion Models
- 2026When Do Diffusion Models learn to Generate Multiple Objects?
- 2024TALE: Training-free Cross-domain Image Composition via Adaptive Latent Manipulation and Energy-guided Optimization
- 2023Collage Diffusion