# Image Captioning with Context-Aware Auxiliary Guidance

arXiv: https://arxiv.org/abs/2012.05545 · HTML: https://mlsanity.com/paper/2012.05545/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [Controllable Image Captioning via Prompting](https://mlsanity.com/paper/2212.01803/) (2022)
- [Efficient Image Captioning for Edge Devices](https://mlsanity.com/paper/2212.08985/) (2022)
- [Image Captioning via Compact Bidirectional Architecture](https://mlsanity.com/paper/2201.01984/) (2022)
- [Efficient Modeling of Future Context for Image Captioning](https://mlsanity.com/paper/2207.10897/) (2022)
- [Self-Annotated Training for Controllable Image Captioning](https://mlsanity.com/paper/2110.08446/) (2021)
- [Bidirectional Awareness Induction in Autoregressive Seq2Seq Models](https://mlsanity.com/paper/2408.13959/) (2024)
- [FODA-PG for Enhanced Medical Imaging Narrative Generation: Adaptive Differentiation of Normal and Abnormal Attributes](https://mlsanity.com/paper/2409.03947/) (2024)
- [Cross Modification Attention Based Deliberation Model for Image Captioning](https://mlsanity.com/paper/2109.08411/) (2021)
- [Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis](https://mlsanity.com/paper/2408.04909/) (2024)
- [Improving Image Captioning with Control Signal of Sentence Quality](https://mlsanity.com/paper/2206.03196/) (2022)

## Abstract (arXiv)

Image captioning is a challenging computer vision task, which aims to generate a natural language description of an image. Most recent researches follow the encoder-decoder framework which depends heavily on the previous generated words for the current prediction. Such methods can not effectively take advantage of the future predicted information to learn complete semantics. In this paper, we propose Context-Aware Auxiliary Guidance (CAAG) mechanism that can guide the captioning model to perceive global contexts. Upon the captioning model, CAAG performs semantic attention that selectively concentrates on useful information of the global predictions to reproduce the current generation. To validate the adaptability of the method, we apply CAAG to three popular captioners and our proposal achieves competitive performance on the challenging Microsoft COCO image captioning benchmark, e.g. 132.2 CIDEr-D score on Karpathy split and 130.7 CIDEr-D (c40) score on official online evaluation server.