# TextPainter: Multimodal Text Image Generation with Visual-harmony and Text-comprehension for Poster Design

arXiv: https://arxiv.org/abs/2308.04733 · HTML: https://mlsanity.com/paper/2308.04733/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese](https://mlsanity.com/paper/2211.01335/) (2022)
- [Multi-Content GAN for Few-Shot Font Style Transfer](https://mlsanity.com/paper/1712.00516/) (2017)
- [Editing Text in the Wild](https://mlsanity.com/paper/1908.03047/) (2019)
- [DG-Font: Deformable Generative Networks for Unsupervised Font Generation](https://mlsanity.com/paper/2104.03064/) (2021)
- [Composition-aware Graphic Layout GAN for Visual-textual Presentation Designs](https://mlsanity.com/paper/2205.00303/) (2022)
- [SwapText: Image Based Texts Transfer in Scenes](https://mlsanity.com/paper/2003.08152/) (2020)
- [TextStyleBrush: Transfer of Text Aesthetics from a Single Example](https://mlsanity.com/paper/2106.08385/) (2021)
- [Few-Shot Font Generation by Learning Fine-Grained Local Styles](https://mlsanity.com/paper/2205.09965/) (2022)
- [Geometry Aligned Variational Transformer for Image-conditioned Layout Generation](https://mlsanity.com/paper/2209.00852/) (2022)
- [Auto-Encoder Guided GAN for Chinese Calligraphy Synthesis](https://mlsanity.com/paper/1706.08789/) (2017)
- [XMP-Font: Self-Supervised Cross-Modality Pre-training for Few-Shot Font Generation](https://mlsanity.com/paper/2204.05084/) (2022)
- [CapOnImage: Context-driven Dense-Captioning on Image](https://mlsanity.com/paper/2204.12974/) (2022)
- [Self-Supervised Text Erasing with Controllable Image Synthesis](https://mlsanity.com/paper/2204.12743/) (2022)
- [APRNet: Attention-based Pixel-wise Rendering Network for Photo-Realistic Text Image Generation](https://mlsanity.com/paper/2203.07705/) (2022)

## Built on by (most cited first, 18 of 18)

- [PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Rendering](https://mlsanity.com/paper/2504.06632/) (2025)
- [Graphic Design with Large Multimodal Model](https://mlsanity.com/paper/2404.14368/) (2024)
- [CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design](https://mlsanity.com/paper/2505.19114/) (2025)
- [From Elements to Design: A Layered Approach for Automatic Graphic Design Composition](https://mlsanity.com/paper/2412.19712/) (2024)
- [Towards Diverse and Consistent Typography Generation](https://mlsanity.com/paper/2309.02099/) (2023)
- [Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editing](https://mlsanity.com/paper/2405.04496/) (2024)
- [Rethinking Layered Graphic Design Generation with a Top-Down Approach](https://mlsanity.com/paper/2507.05601/) (2025)
- [Intelligent Artistic Typography: A Comprehensive Review of Artistic Text Design and Generation](https://mlsanity.com/paper/2407.14774/) (2024)
- [InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generation](https://mlsanity.com/paper/2603.05898/) (2026)
- [Typographic Text Generation with Off-the-Shelf Diffusion Model](https://mlsanity.com/paper/2402.14314/) (2024)
- [VitaGlyph: Vitalizing Artistic Typography with Flexible Dual-branch Diffusion Models](https://mlsanity.com/paper/2410.01738/) (2024)
- [OmniText: A Training-Free Generalist for Controllable Text-Image Manipulation](https://mlsanity.com/paper/2510.24093/) (2025)
- [LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing](https://mlsanity.com/paper/2503.21541/) (2025)
- [COP-GEN-Beta: Unified Generative Modelling of COPernicus Imagery Thumbnails](https://mlsanity.com/paper/2504.08548/) (2025)
- [PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow](https://mlsanity.com/paper/2603.25738/) (2026)
- [PRISM: Learning Design Knowledge from Data for Stylistic Design Improvement](https://mlsanity.com/paper/2601.11747/) (2026)
- [CPT: Controllable and Editable Design Variations with Language Models](https://mlsanity.com/paper/2604.04380/) (2026)
- [Reading Between the Pixels: An Inscriptive Jailbreak Attack on Text-to-Image Models](https://mlsanity.com/paper/2604.05853/) (2026)

## Abstract (arXiv)

Text design is one of the most critical procedures in poster design, as it relies heavily on the creativity and expertise of humans to design text images considering the visual harmony and text-semantic. This study introduces TextPainter, a novel multimodal approach that leverages contextual visual information and corresponding text semantics to generate text images. Specifically, TextPainter takes the global-local background image as a hint of style and guides the text image generation with visual harmony. Furthermore, we leverage the language model and introduce a text comprehension module to achieve both sentence-level and word-level style variations. Besides, we construct the PosterT80K dataset, consisting of about 80K posters annotated with sentence-level bounding boxes and text contents. We hope this dataset will pave the way for further research on multimodal text image generation. Extensive quantitative and qualitative experiments demonstrate that TextPainter can generate visually-and-semantically-harmonious text images for posters.