# Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poems

arXiv: https://arxiv.org/abs/2109.11682 · HTML: https://mlsanity.com/paper/2109.11682/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [Transferring GANs: generating images from limited data](https://mlsanity.com/paper/1805.01677/) (2018)
- [An Introduction to Image Synthesis with Generative Adversarial Nets](https://mlsanity.com/paper/1803.04469/) (2018)
- [Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training](https://mlsanity.com/paper/1804.08473/) (2018)
- [How Images Inspire Poems: Generating Classical Chinese Poetry from Images with Memory Networks](https://mlsanity.com/paper/1803.02994/) (2018)
- [Image Inspired Poetry Generation in XiaoIce](https://mlsanity.com/paper/1808.03090/) (2018)
- [A Multi-Modal Chinese Poetry Generation Model](https://mlsanity.com/paper/1806.09792/) (2018)

## Built on by (most cited first, 18 of 18)

- [Diffusion-Based Visual Art Creation: A Survey and New Perspectives](https://mlsanity.com/paper/2408.12128/) (2024)
- [Text-to-Image Synthesis: A Decade Survey](https://mlsanity.com/paper/2411.16164/) (2024)
- [ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding](https://mlsanity.com/paper/2505.06020/) (2025)
- [Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuning](https://mlsanity.com/paper/2508.04848/) (2025)
- [A Generic Method for Fine-grained Category Discovery in Natural Language Texts](https://mlsanity.com/paper/2406.13103/) (2024)
- [Ada-HGNN: Adaptive Sampling for Scalable Hypergraph Neural Networks](https://mlsanity.com/paper/2405.13372/) (2024)
- [Using Causality for Enhanced Prediction of Web Traffic Time Series](https://mlsanity.com/paper/2502.00612/) (2025)
- [Poetry in Pixels: Prompt Tuning for Poem Image Generation via Diffusion Models](https://mlsanity.com/paper/2501.05839/) (2025)
- [PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement](https://mlsanity.com/paper/2507.13708/) (2025)
- [Controllable Stylistic Text Generation with Train-Time Attribute-Regularized Diffusion](https://mlsanity.com/paper/2510.06386/) (2025)
- [Structured Information for Improving Spatial Relationships in Text-to-Image Generation](https://mlsanity.com/paper/2509.15962/) (2025)
- [Understanding Network Behaviors through Natural Language Question-Answering](https://mlsanity.com/paper/2510.21894/) (2025)
- [A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding](https://mlsanity.com/paper/2604.19689/) (2026)
- [DeepArt: A Benchmark to Advance Fidelity Research in AI-Generated Content](https://mlsanity.com/paper/2312.10407/) (2023)
- [Automated Prompt Generation for Creative and Counterfactual Text-to-image Synthesis](https://mlsanity.com/paper/2509.21375/) (2025)
- [MCGA: A Multi-task Classical Chinese Literary Genre Audio Corpus](https://mlsanity.com/paper/2601.09270/) (2026)
- [Reviving Ancient Paintings via Poem: A Colorization Framework for Aligning Cultural Semantics](https://mlsanity.com/paper/2607.17638/) (2026)
- [MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding](https://mlsanity.com/paper/2608.10706/) (2026)

## Abstract (arXiv)

In this work we propose a new task: artistic visualization of classical Chinese poems, where the goal is to generatepaintings of a certain artistic style for classical Chinese poems. For this purpose, we construct a new dataset called Paint4Poem. Thefirst part of Paint4Poem consists of 301 high-quality poem-painting pairs collected manually from an influential modern Chinese artistFeng Zikai. As its small scale poses challenges for effectively training poem-to-painting generation models, we introduce the secondpart of Paint4Poem, which consists of 3,648 caption-painting pairs collected manually from Feng Zikai's paintings and 89,204 poem-painting pairs collected automatically from the web. We expect the former to help learning the artist painting style as it containshis most paintings, and the latter to help learning the semantic relevance between poems and paintings. Further, we analyze Paint4Poem regarding poem diversity, painting style, and the semantic relevance between poems and paintings. We create abenchmark for Paint4Poem: we train two representative text-to-image generation models: AttnGAN and MirrorGAN, and evaluate theirperformance regarding painting pictorial quality, painting stylistic relevance, and semantic relevance between poems and paintings.The results indicate that the models are able to generate paintings that have good pictorial quality and mimic Feng Zikai's style, but thereflection of poem semantics is limited. The dataset also poses many interesting research directions on this task, including transferlearning, few-shot learning, text-to-image generation for low-resource data etc. The dataset is publicly available.(https://github.com/paint4poem/paint4poem)