TextGround4M: A Prompt-Aligned Dataset for Layout-Aware Text Rendering
Not on the map. Why?
Abstract (arXiv)
Despite recent advances in text-to-image generation, models still struggle to accurately render prompt-specified text with correct spatial layout -- especially in multi-span, structured settings. This challenge is driven not only by the lack of datasets that align prompts with the exact text and layout expected in the image, but also by the absence of effective metrics for evaluating layout quality. To address these issues, we introduce TextGround4M, a large-scale dataset of over 4 million prompt-image pairs, each annotated with span-level text grounded in the prompt and corresponding bounding boxes. This enables fine-grained supervision for layout-aware, prompt-grounded text rendering. Building on this, we propose a lightweight training strategy for autoregressive T2I models that appends layout-aware span tokens during training, without altering model architecture or inference behavior. We further construct a benchmark with stratified layout complexity to evaluate both open-source and proprietary models in a zero-shot setting. In addition, we introduce two layout-aware metrics to address the long-standing lack of spatial evaluation in text rendering. Our results show that models trained on TextGround4M outperform strong baselines in text fidelity, spatial accuracy, and prompt consistency, highlighting the importance of fine-grained layout supervision for grounded T2I generation.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
19 of 29 references.
References · 19
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
- 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
- 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
- 2023TextDiffuser: Diffusion Models as Text Painterscited by 223
7 more
- 2023AnyText: Multilingual Visual Text Generation And Editingcited by 165
- 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
- 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 141
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 136
- 2023GlyphControl: Glyph Conditional Control for Visual Text Generationcited by 127
- 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
- 2025Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modelingcited by 35
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Renderingcited by 1
- 2026VectorHarness: Recovering Editable, Relation-Preserving Structure from Scientific Graphicscited by 0
- 2026Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generationcited by 0
- 2026Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation19 Sep 2026
- 2026VectorHarness: Recovering Editable, Relation-Preserving Structure from Scientific Graphics9 Sep 2026
- 2026Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering1 Jun 2026