# Fast Image Caption Generation with Position Alignment

arXiv: https://arxiv.org/abs/1912.06365 · HTML: https://mlsanity.com/paper/1912.06365/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2019.

## Built on by (most cited first, 21 of 21)

- [From Show to Tell: A Survey on Deep Learning-based Image Captioning](https://mlsanity.com/paper/2107.06912/) (2021)
- [SkyReels-A2: Compose Anything in Video Diffusion Transformers](https://mlsanity.com/paper/2504.02436/) (2025)
- [Scaling Diffusion Transformers to 16 Billion Parameters](https://mlsanity.com/paper/2407.11633/) (2024)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [A-JEPA: Joint-Embedding Predictive Architecture Can Listen](https://mlsanity.com/paper/2311.15830/) (2023)
- [Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models](https://mlsanity.com/paper/2404.04478/) (2024)
- [FLUX that Plays Music](https://mlsanity.com/paper/2409.00587/) (2024)
- [Exploring Discrete Diffusion Models for Image Captioning](https://mlsanity.com/paper/2211.11694/) (2022)
- [Ingredients: Blending Custom Photos with Video Diffusion Transformers](https://mlsanity.com/paper/2501.01790/) (2025)
- [Semi-Autoregressive Transformer for Image Captioning](https://mlsanity.com/paper/2106.09436/) (2021)
- [Video Diffusion Transformers are In-Context Learners](https://mlsanity.com/paper/2412.10783/) (2024)
- [Semi-Autoregressive Image Captioning](https://mlsanity.com/paper/2110.05342/) (2021)
- [LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?](https://mlsanity.com/paper/2404.10763/) (2024)
- [Explicit Image Caption Editing](https://mlsanity.com/paper/2207.09625/) (2022)
- [Music Consistency Models](https://mlsanity.com/paper/2404.13358/) (2024)
- [Uncertainty-Aware Image Captioning](https://mlsanity.com/paper/2211.16769/) (2022)
- [Efficient Modeling of Future Context for Image Captioning](https://mlsanity.com/paper/2207.10897/) (2022)
- [DiffCap: Exploring Continuous Diffusion on Image Captioning](https://mlsanity.com/paper/2305.12144/) (2023)
- [Fast Sequence Generation with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2101.09698/) (2021)
- [Bounding and Filling: A Fast and Flexible Framework for Image Captioning](https://mlsanity.com/paper/2310.09876/) (2023)
- [Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Image](https://mlsanity.com/paper/2410.15312/) (2024)

## Abstract (arXiv)

Recent neural network models for image captioning usually employ an encoder-decoder architecture, where the decoder adopts a recursive sequence decoding way. However, such autoregressive decoding may result in sequential error accumulation and slow generation which limit the applications in practice. Non-autoregressive (NA) decoding has been proposed to cover these issues but suffers from language quality problem due to the indirect modeling of the target distribution. Towards that end, we propose an improved NA prediction framework to accelerate image captioning. Our decoding part consists of a position alignment to order the words that describe the content detected in the given image, and a fine non-autoregressive decoder to generate elegant descriptions. Furthermore, we introduce an inference strategy that regards position information as a latent variable to guide the further sentence generation. The Experimental results on public datasets show that our proposed model achieves better performance compared to general NA captioning models, while achieves comparable performance as autoregressive image captioning models with a significant speedup.