mlsanityData to 14 Oct 2026

TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation

arXiv 2404.11824 ↗First posted 18 Apr 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Text-to-image (T2I) generation has made remarkable progress in producing high-quality images, but a fundamental challenge remains: creating backgrounds that naturally accommodate text placement without compromising image quality. This capability is non-trivial for real-world applications like graphic design, where clear visual hierarchy between content and text is essential. Prior work has primarily focused on arranging layouts within existing static images, leaving unexplored the potential of T2I models for generating text-friendly backgrounds. We present TextCenGen, a training-free dynamic background adaptation in the blank region for text-friendly image generation. Instead of directly reducing attention in text areas, which degrades image quality, we relocate conflicting objects before background optimization. Our method analyzes cross-attention maps to identify conflicting objects overlapping with text regions and uses a force-directed graph approach to guide their relocation, followed by attention excluding constraints to ensure smooth backgrounds. Our method is plug-and-play, requiring no additional training while well balancing both semantic fidelity and visual quality. Evaluated on our proposed text-friendly T2I benchmark of 27,000 images across four seed datasets, TextCenGen outperforms existing methods by achieving 23% lower saliency overlap in text regions while maintaining 98% of the semantic fidelity measured by CLIP score and our proposed Visual-Textual Concordance Metric (VTCM).

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

43 of 58 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
31 more
  1. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  3. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  4. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  5. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  6. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  7. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 359
  8. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  9. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
  10. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
  11. 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
  12. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  13. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 212
  14. 2023AnyText: Multilingual Visual Text Generation And Editingcited by 170
  15. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 153
  16. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
  17. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Modelscited by 142
  18. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
  19. 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
  20. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  21. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 91
  22. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Modelscited by 83
  23. 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
  24. 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
  25. 2021Unifying Multimodal Transformer for Bi-directional Image and Text Generationcited by 41
  26. 2023Unifying Layout Generation with a Decoupled Diffusion Modelcited by 41
  27. 2023Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Modelcited by 41
  28. 2023Relation-Aware Diffusion Model for Controllable Poster Layout Generationcited by 37
  29. 2023Towards Flexible Multi-modal Document Modelscited by 31
  30. 2024Desigen: A Pipeline for Controllable Design Template Generationcited by 19
  31. 2023Text2Poster: Laying out Stylized Texts on Retrieved Imagescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.