# Recurrent Topic-Transition GAN for Visual Paragraph Generation

arXiv: https://arxiv.org/abs/1703.07022 · HTML: https://mlsanity.com/paper/1703.07022/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2017.

## Built on by (most cited first, 25 of 25)

- [Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation](https://mlsanity.com/paper/2106.06963/) (2021)
- [From Show to Tell: A Survey on Deep Learning-based Image Captioning](https://mlsanity.com/paper/2107.06912/) (2021)
- [Competence-based Multimodal Curriculum Learning for Medical Report Generation](https://mlsanity.com/paper/2206.14579/) (2022)
- [Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures](https://mlsanity.com/paper/2307.15220/) (2023)
- [KiUT: Knowledge-injected U-Transformer for Radiology Report Generation](https://mlsanity.com/paper/2306.11345/) (2023)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation](https://mlsanity.com/paper/2406.11451/) (2024)
- [Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generation](https://mlsanity.com/paper/2303.15932/) (2023)
- [Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning](https://mlsanity.com/paper/2206.01843/) (2022)
- [IC3: Image Captioning by Committee Consensus](https://mlsanity.com/paper/2302.01328/) (2023)
- [A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot](https://mlsanity.com/paper/2305.09758/) (2023)
- [What Makes for Good Image Captions?](https://mlsanity.com/paper/2405.00485/) (2024)
- [REMAST: Real-time Emotion-based Music Arrangement with Soft Transition](https://mlsanity.com/paper/2305.08029/) (2023)
- [Topicwise Separable Sentence Retrieval for Medical Report Generation](https://mlsanity.com/paper/2405.04175/) (2024)
- [CLID: Controlled-Length Image Descriptions with Limited Data](https://mlsanity.com/paper/2211.14835/) (2022)
- [Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning](https://mlsanity.com/paper/2105.04143/) (2021)
- [Attacks and Defenses for Free-Riders in Multi-Discriminator GAN](https://mlsanity.com/paper/2201.09967/) (2022)
- [VLIS: Unimodal Language Models Guide Multimodal Language Generation](https://mlsanity.com/paper/2310.09767/) (2023)
- [Enhanced Modality Transition for Image Captioning](https://mlsanity.com/paper/2102.11526/) (2021)
- [Describing image focused in cognitive and visual details for visually impaired people: An approach to generating inclusive paragraphs](https://mlsanity.com/paper/2202.05331/) (2022)
- [Bypass Network for Semantics Driven Image Paragraph Captioning](https://mlsanity.com/paper/2206.10059/) (2022)
- [IIHT: Medical Report Generation with Image-to-Indicator Hierarchical Transformer](https://mlsanity.com/paper/2308.05633/) (2023)
- [A Fine-Grained Image Description Generation Method Based on Joint Objectives](https://mlsanity.com/paper/2311.12799/) (2023)
- [Intensive Vision-guided Network for Radiology Report Generation](https://mlsanity.com/paper/2402.03754/) (2024)
- [EIR: Enhanced Image Representations for Medical Report Generation](https://mlsanity.com/paper/2512.23185/) (2025)

## Abstract (arXiv)

A natural image usually conveys rich semantic content and can be viewed from different angles. Existing image description methods are largely restricted by small sets of biased visual paragraph annotations, and fail to cover rich underlying semantics. In this paper, we investigate a semi-supervised paragraph generative framework that is able to synthesize diverse and semantically coherent paragraph descriptions by reasoning over local semantic regions and exploiting linguistic knowledge. The proposed Recurrent Topic-Transition Generative Adversarial Network (RTT-GAN) builds an adversarial framework between a structured paragraph generator and multi-level paragraph discriminators. The paragraph generator generates sentences recurrently by incorporating region-based visual and language attention mechanisms at each step. The quality of generated paragraph sentences is assessed by multi-level adversarial discriminators from two aspects, namely, plausibility at sentence level and topic-transition coherence at paragraph level. The joint adversarial training of RTT-GAN drives the model to generate realistic paragraphs with smooth logical transition between sentence topics. Extensive quantitative experiments on image and video paragraph datasets demonstrate the effectiveness of our RTT-GAN in both supervised and semi-supervised settings. Qualitative results on telling diverse stories for an image also verify the interpretability of RTT-GAN.