# Image Inspired Poetry Generation in XiaoIce

arXiv: https://arxiv.org/abs/1808.03090 · HTML: https://mlsanity.com/paper/1808.03090/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2018.

## Built on by (most cited first, 6 of 6)

- [Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poems](https://mlsanity.com/paper/2109.11682/) (2021)
- [Lingxi: A Diversity-aware Chinese Modern Poetry Generation System](https://mlsanity.com/paper/2108.12108/) (2021)
- [Multi-Modal Experience Inspired AI Creation](https://mlsanity.com/paper/2209.02427/) (2022)
- [Poetry in Pixels: Prompt Tuning for Poem Image Generation via Diffusion Models](https://mlsanity.com/paper/2501.05839/) (2025)
- [PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement](https://mlsanity.com/paper/2507.13708/) (2025)
- [Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs](https://mlsanity.com/paper/2605.22654/) (2026)

## Abstract (arXiv)

Vision is a common source of inspiration for poetry. The objects and the sentimental imprints that one perceives from an image may lead to various feelings depending on the reader. In this paper, we present a system of poetry generation from images to mimic the process. Given an image, we first extract a few keywords representing objects and sentiments perceived from the image. These keywords are then expanded to related ones based on their associations in human written poems. Finally, verses are generated gradually from the keywords using recurrent neural networks trained on existing poems. Our approach is evaluated by human assessors and compared to other generation baselines. The results show that our method can generate poems that are more artistic than the baseline methods. This is one of the few attempts to generate poetry from images. By deploying our proposed approach, XiaoIce has already generated more than 12 million poems for users since its release in July 2017. A book of its poems has been published by Cheers Publishing, which claimed that the book is the first-ever poetry collection written by an AI in human history.