TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation
Not on the map. Why?
Abstract (arXiv)
Text-to-image (T2I) generation has made remarkable progress in producing high-quality images, but a fundamental challenge remains: creating backgrounds that naturally accommodate text placement without compromising image quality. This capability is non-trivial for real-world applications like graphic design, where clear visual hierarchy between content and text is essential. Prior work has primarily focused on arranging layouts within existing static images, leaving unexplored the potential of T2I models for generating text-friendly backgrounds. We present TextCenGen, a training-free dynamic background adaptation in the blank region for text-friendly image generation. Instead of directly reducing attention in text areas, which degrades image quality, we relocate conflicting objects before background optimization. Our method analyzes cross-attention maps to identify conflicting objects overlapping with text regions and uses a force-directed graph approach to guide their relocation, followed by attention excluding constraints to ensure smooth backgrounds. Our method is plug-and-play, requiring no additional training while well balancing both semantic fidelity and visual quality. Evaluated on our proposed text-friendly T2I benchmark of 27,000 images across four seed datasets, TextCenGen outperforms existing methods by achieving 23% lower saliency overlap in text regions while maintaining 98% of the semantic fidelity measured by CLIP score and our proposed Visual-Textual Concordance Metric (VTCM).
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 18 Apr 2024.
Builds on
43 of 58 references.
References · 43
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
31 more
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
- 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
- 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
- 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 359
- 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
- 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
- 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
- 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
- 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
- 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 212
- 2023AnyText: Multilingual Visual Text Generation And Editingcited by 170
- 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 153
- 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
- 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Modelscited by 142
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
- 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
- 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
- 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 91
- 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Modelscited by 83
- 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
- 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
- 2021Unifying Multimodal Transformer for Bi-directional Image and Text Generationcited by 41
- 2023Unifying Layout Generation with a Decoupled Diffusion Modelcited by 41
- 2023Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Modelcited by 41
- 2023Relation-Aware Diffusion Model for Controllable Poster Layout Generationcited by 37
- 2023Towards Flexible Multi-modal Document Modelscited by 31
- 2024Desigen: A Pipeline for Controllable Design Template Generationcited by 19
- 2023Text2Poster: Laying out Stylized Texts on Retrieved Imagescited by 5
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCRcited by 12
- 2025Rethinking Layered Graphic Design Generation with a Top-Down Approachcited by 10
- 2026Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generationcited by 0
- 2026Planning and Rendering in Concert: DeepFusion of Autoregressive Layouts and Diffusion for Visual Text Generation19 Sep 2026
- 2025Rethinking Layered Graphic Design Generation with a Top-Down Approach8 Jul 2025
- 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR15 Apr 2025