# Improving Reinforcement Learning Based Image Captioning with Natural Language Prior

arXiv: https://arxiv.org/abs/1809.06227 · HTML: https://mlsanity.com/paper/1809.06227/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2018.

## Built on by (most cited first, 4 of 4)

- [ACORT: A Compact Object Relation Transformer for Parameter Efficient Image Captioning](https://mlsanity.com/paper/2202.05451/) (2022)
- [ALCAP: Alignment-Augmented Music Captioner](https://mlsanity.com/paper/2212.10901/) (2022)
- [Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis](https://mlsanity.com/paper/2408.04909/) (2024)
- [RL-MoE: An Image-Based Privacy Preserving Approach In Intelligent Transportation System](https://mlsanity.com/paper/2508.09186/) (2025)

## Abstract (arXiv)

Recently, Reinforcement Learning (RL) approaches have demonstrated advanced performance in image captioning by directly optimizing the metric used for testing. However, this shaped reward introduces learning biases, which reduces the readability of generated text. In addition, the large sample space makes training unstable and slow. To alleviate these issues, we propose a simple coherent solution that constrains the action space using an n-gram language prior. Quantitative and qualitative evaluations on benchmarks show that RL with the simple add-on module performs favorably against its counterpart in terms of both readability and speed of convergence. Human evaluation results show that our model is more human readable and graceful. The implementation will become publicly available upon the acceptance of the paper.