# ADS-Cap: A Framework for Accurate and Diverse Stylized Captioning with Unpaired Stylistic Corpora

arXiv: https://arxiv.org/abs/2308.01143 · HTML: https://mlsanity.com/paper/2308.01143/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning](https://mlsanity.com/paper/1612.01887/) (2016)
- [SentiCap: Generating Image Descriptions with Sentiments](https://mlsanity.com/paper/1510.01431/) (2015)
- [Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space](https://mlsanity.com/paper/1711.07068/) (2017)
- [SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text](https://mlsanity.com/paper/1805.07030/) (2018)
- ["Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention](https://mlsanity.com/paper/1807.03871/) (2018)
- [Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning](https://mlsanity.com/paper/1908.08529/) (2019)
- [Image Captioning at Will: A Versatile Scheme for Effectively Injecting Sentiments into Image Descriptions](https://mlsanity.com/paper/1801.10121/) (2018)
- [Unsupervised Stylish Image Description Generation via Domain Layer Norm](https://mlsanity.com/paper/1809.06214/) (2018)
- [Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioning](https://mlsanity.com/paper/2108.11912/) (2021)
- [Towards Generating Stylized Image Captions via Adversarial Training](https://mlsanity.com/paper/1908.02943/) (2019)

## Built on by (most cited first, 1 of 1)

- [CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era](https://mlsanity.com/paper/2503.12329/) (2025)

## Abstract (arXiv)

Generating visually grounded image captions with specific linguistic styles using unpaired stylistic corpora is a challenging task, especially since we expect stylized captions with a wide variety of stylistic patterns. In this paper, we propose a novel framework to generate Accurate and Diverse Stylized Captions (ADS-Cap). Our ADS-Cap first uses a contrastive learning module to align the image and text features, which unifies paired factual and unpaired stylistic corpora during the training process. A conditional variational auto-encoder is then used to automatically memorize diverse stylistic patterns in latent space and enhance diversity through sampling. We also design a simple but effective recheck module to boost style accuracy by filtering style-specific captions. Experimental results on two widely used stylized image captioning datasets show that regarding consistency with the image, style accuracy and diversity, ADS-Cap achieves outstanding performances compared to various baselines. We finally conduct extensive analyses to understand the effectiveness of our method. Our code is available at https://github.com/njucckevin/ADS-Cap.