Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation
Not on the map. Why?
Abstract (arXiv)
Generating text-rich images from prompts requires both textual fidelity and the coherent integration of text into the surrounding image. An explicit layout can provide structured guidance about what text should appear and where, but a well-formed plan alone does not guarantee that the renderer will realize it faithfully. Existing layout-based AR-diffusion systems typically optimize planning and rendering separately, preventing the planner's representations from being adapted jointly with image synthesis. We introduce DuetGen, an autonomous visual text generator built on DeepFusion, which jointly learns autoregressive planning and continuous diffusion rendering. DeepFusion conditions a diffusion transformer on the planner's prompt and bbox-content hidden states, allowing rendering supervision to shape the representations connecting textual plans with visual outputs. Its joint objective combines autoregressive plan supervision, text-region-weighted diffusion learning, and auxiliary coordinate supervision to maintain structured planning, emphasize text-bearing regions, and improve the spatial precision of planner representations. During inference, Phase-Aware Attention Modulation strengthens the correspondence between image regions and their matched coordinate and content states, facilitating region-specific execution of the generated plan. With a 2B planner and a 4B single-stream DiT, DuetGen achieves 0.8293 word accuracy on CVTG-2K and 0.938 accuracy on LongText-Bench, closely matching the substantially larger Qwen-Image on both benchmarks. These results demonstrate the value of jointly learned planning representations and region-specific rendering for autonomous visual text generation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 71 of its 78 references that are in the map. First posted on arXiv 19 Sep 2026.
Builds on
71 of 78 references.
References · 71
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
- 2025Qwen-Image Technical Reportcited by 1,044
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
38 more
- 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
- 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
- 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
- 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
- 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
- 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
- 2023AnyText: Multilingual Visual Text Generation And Editingcited by 170
- 2025Seedream 3.0 Technical Reportcited by 154
- 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 153
- 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 141
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
- 2023GlyphControl: Glyph Conditional Control for Visual Text Generationcited by 131
- 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 124
- 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 112
- 2024Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Renderingcited by 81
- 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 60
- 2023UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Modelscited by 59
- 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 54
- 2024AnyText2: Visual Text Generation and Editing With Customizable Attributescited by 53
- 2024Glyph-ByT5-v2: A Strong Aesthetic Baseline for Accurate Multilingual Visual Text Renderingcited by 48
- 2025PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Renderingcited by 46
- 2023Improving Diffusion Models for Scene Text Editing with Dual Encoderscited by 45
- 2023Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Modelcited by 41
- 2024FonTS: Text Rendering with Typography and Style Controlscited by 40
- 2025DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Modelscited by 40
- 2025PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Frameworkcited by 40
- 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 37
- 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
- 2023GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generationcited by 34
- 2024GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Modelscited by 29
- 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
- 2025TextAtlas5M: A Large-scale Dataset for Dense Text Image Generationcited by 27
- 2025MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generationcited by 23
- 2025BizGen: Advancing Article-level Visual Text Rendering for Infographics Generationcited by 20
50 most cited of 71 All 71 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MammothModa2: A Unified AR-Diffusion Framework for Multimodal Understanding and Generation
- 2025Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model
- 2024Training-free Composite Scene Generation for Layout-to-Image Synthesis
- 2026Layer-wise Instance Binding for Regional and Occlusion Control in Text-to-Image Diffusion Transformers
- 2025Visual Autoregressive Models Beat Diffusion Models on Inference Time Scaling
- 2025A Two-Stage System for Layout-Controlled Image Generation using Large Language Models and Diffusion Models
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
- 2026Beyond Text Conditioning: A Systematic Study of MLLM-DiT Fusion for Video Generation