# Analysis of diversity-accuracy tradeoff in image captioning

arXiv: https://arxiv.org/abs/2002.11848 · HTML: https://mlsanity.com/paper/2002.11848/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [Variational Topic Inference for Chest X-Ray Report Generation](https://mlsanity.com/paper/2107.07314/) (2021)
- [Group-based Distinctive Image Captioning with Memory Attention](https://mlsanity.com/paper/2108.09151/) (2021)
- [Learning Distinct and Representative Styles for Image Captioning](https://mlsanity.com/paper/2209.08231/) (2022)
- [CaptionSmiths: Flexibly Controlling Language Pattern in Image Captioning](https://mlsanity.com/paper/2507.01409/) (2025)
- [Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning](https://mlsanity.com/paper/2212.03230/) (2022)
- [Variational Transformer: A Framework Beyond the Trade-off between Accuracy and Diversity for Image Captioning](https://mlsanity.com/paper/2205.14458/) (2022)
- [Group-based Distinctive Image Captioning with Memory Difference Encoding and Attention](https://mlsanity.com/paper/2504.02496/) (2025)

## Abstract (arXiv)

We investigate the effect of different model architectures, training objectives, hyperparameter settings and decoding procedures on the diversity of automatically generated image captions. Our results show that 1) simple decoding by naive sampling, coupled with low temperature is a competitive and fast method to produce diverse and accurate caption sets; 2) training with CIDEr-based reward using Reinforcement learning harms the diversity properties of the resulting generator, which cannot be mitigated by manipulating decoding parameters. In addition, we propose a new metric AllSPICE for evaluating both accuracy and diversity of a set of captions by a single value.