mlsanityData to 7 Oct 2026

CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-step

arXiv 2507.04451 ↗First posted 6 Jul 2025cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Current text-to-image (T2I) generation models struggle to align spatial composition with the input text, especially in complex scenes. Even layout-based approaches yield suboptimal spatial control, as their generation process is decoupled from layout planning, making it difficult to refine the layout during synthesis. We present CoT-Diff, a framework that brings step-by-step CoT-style reasoning into T2I generation by tightly integrating Multimodal Large Language Model (MLLM)-driven 3D layout planning with the diffusion process. CoT-Diff enables layout-aware reasoning inline within a single diffusion round: at each denoising step, the MLLM evaluates intermediate predictions, dynamically updates the 3D scene layout, and continuously guides the generation process. The updated layout is converted into semantic conditions and depth maps, which are fused into the diffusion model via a condition-aware attention mechanism, enabling precise spatial control and semantic injection. Experiments on 3D Scene benchmarks show that CoT-Diff significantly improves spatial alignment and compositional fidelity, and outperforms the state-of-the-art method by 34.7% in complex scene spatial accuracy, thereby validating the effectiveness of this entangled generation paradigm.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 50 of its 55 references that are in the map. First posted on arXiv 6 Jul 2025.

Builds on

50 of 55 references.

References · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2023Segment Anythingcited by 8,158
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2025Qwen2.5-VL Technical Reportcited by 5,418
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  5. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  7. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  8. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  9. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  10. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  11. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  12. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  13. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
  14. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  15. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
  16. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  17. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  18. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 169
  19. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 163
  20. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  21. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  22. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
  23. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
  24. 2021Simple multi-dataset detectioncited by 126
  25. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  26. 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
  27. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  28. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
  29. 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 73
  30. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  31. 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
  32. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 36
  33. 2025EliGen: Entity-Level Controlled Image Generation with Regional Attentioncited by 32
  34. 2024Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidancecited by 30
  35. 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
  36. 20253DIS-FLUX: simple and efficient multi-instance generation with DiT renderingcited by 19
  37. 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generationcited by 18
  38. 2024Draw Like an Artist: Complex Scene Generation with Diffusion Model via Composition, Painting, and Retouchingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024DivCon: Divide and Conquer for Complex Numerical and Spatial Reasoning in Text-to-Image Generation
  2. 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
  3. 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
  4. 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation
  5. 2026Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings
  6. 2024Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models
  7. 2026CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation
  8. 2023LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation