CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-step
Not on the map. Why?
Abstract (arXiv)
Current text-to-image (T2I) generation models struggle to align spatial composition with the input text, especially in complex scenes. Even layout-based approaches yield suboptimal spatial control, as their generation process is decoupled from layout planning, making it difficult to refine the layout during synthesis. We present CoT-Diff, a framework that brings step-by-step CoT-style reasoning into T2I generation by tightly integrating Multimodal Large Language Model (MLLM)-driven 3D layout planning with the diffusion process. CoT-Diff enables layout-aware reasoning inline within a single diffusion round: at each denoising step, the MLLM evaluates intermediate predictions, dynamically updates the 3D scene layout, and continuously guides the generation process. The updated layout is converted into semantic conditions and depth maps, which are fused into the diffusion model via a condition-aware attention mechanism, enabling precise spatial control and semantic injection. Experiments on 3D Scene benchmarks show that CoT-Diff significantly improves spatial alignment and compositional fidelity, and outperforms the state-of-the-art method by 34.7% in complex scene spatial accuracy, thereby validating the effectiveness of this entangled generation paradigm.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 50 of its 55 references that are in the map. First posted on arXiv 6 Jul 2025.
Builds on
50 of 55 references.
References · 50
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2023Segment Anythingcited by 8,158
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2025Qwen2.5-VL Technical Reportcited by 5,418
38 more
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
- 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
- 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
- 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
- 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
- 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 169
- 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 163
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
- 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
- 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
- 2021Simple multi-dataset detectioncited by 126
- 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
- 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
- 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
- 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
- 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 73
- 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
- 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
- 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 36
- 2025EliGen: Entity-Level Controlled Image Generation with Regional Attentioncited by 32
- 2024Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidancecited by 30
- 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
- 20253DIS-FLUX: simple and efficient multi-instance generation with DiT renderingcited by 19
- 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generationcited by 18
- 2024Draw Like an Artist: Complex Scene Generation with Diffusion Model via Composition, Painting, and Retouchingcited by 6
Built on by
Citing papers per year: 2025: 4; 2026: 4.
8 papers.
- 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 78
- 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 23
- 2025Diverse Text-to-Image Generation via Contrastive Noise Optimizationcited by 10
- 2026StruVis: Enhancing Reasoning-based Text-to-Image Generation via Thinking with Structured Visioncited by 3
- 2026OctoT2I: A Self-Evolving Agentic Text-to-Image Routercited by 3
- 2026Advancing Aesthetic Image Generation via Composition Transfercited by 2
- 2025FoR-SALE: Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editingcited by 1
- 2026Test-Time Instance-Specific Parameter Composition: A New Paradigm for Adaptive Generative Modelingcited by 0
- 2026OctoT2I: A Self-Evolving Agentic Text-to-Image Router1 Jun 2026
- 2026Advancing Aesthetic Image Generation via Composition Transfer6 May 2026
- 2026Test-Time Instance-Specific Parameter Composition: A New Paradigm for Adaptive Generative Modeling29 Mar 2026
- 2026StruVis: Enhancing Reasoning-based Text-to-Image Generation via Thinking with Structured Vision6 Mar 2026
- 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editor5 Dec 2025
- 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback19 Oct 2025
- 2025Diverse Text-to-Image Generation via Contrastive Noise Optimization4 Oct 2025
- 2025FoR-SALE: Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing27 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2024DivCon: Divide and Conquer for Complex Numerical and Spatial Reasoning in Text-to-Image Generation
- 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
- 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
- 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation
- 2026Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings
- 2024Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models
- 2026CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation
- 2023LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation