# Show, Recall, and Tell: Image Captioning with Recall Mechanism

arXiv: https://arxiv.org/abs/2001.05876 · HTML: https://mlsanity.com/paper/2001.05876/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2020.

## Built on by (most cited first, 27 of 27)

- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [From Show to Tell: A Survey on Deep Learning-based Image Captioning](https://mlsanity.com/paper/2107.06912/) (2021)
- [One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos](https://mlsanity.com/paper/2409.19603/) (2024)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [Dual-Level Collaborative Transformer for Image Captioning](https://mlsanity.com/paper/2101.06462/) (2021)
- [MeaCap: Memory-Augmented Zero-shot Image Captioning](https://mlsanity.com/paper/2403.03715/) (2024)
- [ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing](https://mlsanity.com/paper/2303.02437/) (2023)
- [Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation](https://mlsanity.com/paper/2109.05743/) (2021)
- [Retrieval-augmented Image Captioning](https://mlsanity.com/paper/2302.08268/) (2023)
- [Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning](https://mlsanity.com/paper/2205.04363/) (2022)
- [Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioning](https://mlsanity.com/paper/2208.06551/) (2022)
- [Impossible Videos](https://mlsanity.com/paper/2503.14378/) (2025)
- [Transforming Visual Scene Graphs to Image Captions](https://mlsanity.com/paper/2305.02177/) (2023)
- [Neural Attention for Image Captioning: Review of Outstanding Methods](https://mlsanity.com/paper/2111.15015/) (2021)
- [A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes](https://mlsanity.com/paper/2403.07469/) (2024)
- [Image Captioning via Compact Bidirectional Architecture](https://mlsanity.com/paper/2201.01984/) (2022)
- [Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles](https://mlsanity.com/paper/2211.16504/) (2022)
- [Efficient Modeling of Future Context for Image Captioning](https://mlsanity.com/paper/2207.10897/) (2022)
- [Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach](https://mlsanity.com/paper/2408.07249/) (2024)
- [Image Captioning In the Transformer Age](https://mlsanity.com/paper/2204.07374/) (2022)
- [Exploring the sequence length bottleneck in the Transformer for Image Captioning](https://mlsanity.com/paper/2207.03327/) (2022)
- [Macroscopic Control of Text Generation for Image Captioning](https://mlsanity.com/paper/2101.08000/) (2021)
- [Geometry-Entangled Visual Semantic Transformer for Image Captioning](https://mlsanity.com/paper/2109.14137/) (2021)
- [A Comprehensive Analysis of Real-World Image Captioning and Scene Identification](https://mlsanity.com/paper/2308.02833/) (2023)
- [Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis](https://mlsanity.com/paper/2408.04909/) (2024)
- [EXMODD: An EXplanatory Multimodal Open-Domain Dialogue dataset](https://mlsanity.com/paper/2310.10967/) (2023)
- [Diffusion Is Your Friend in Show, Suggest and Tell](https://mlsanity.com/paper/2512.10038/) (2025)

## Abstract (arXiv)

Generating natural and accurate descriptions in image cap-tioning has always been a challenge. In this paper, we pro-pose a novel recall mechanism to imitate the way human con-duct captioning. There are three parts in our recall mecha-nism : recall unit, semantic guide (SG) and recalled-wordslot (RWS). Recall unit is a text-retrieval module designedto retrieve recalled words for images. SG and RWS are de-signed for the best use of recalled words. SG branch cangenerate a recalled context, which can guide the process ofgenerating caption. RWS branch is responsible for copyingrecalled words to the caption. Inspired by pointing mecha-nism in text summarization, we adopt a soft switch to balancethe generated-word probabilities between SG and RWS. Inthe CIDEr optimization step, we also introduce an individualrecalled-word reward (WR) to boost training. Our proposedmethods (SG+RWS+WR) achieve BLEU-4 / CIDEr / SPICEscores of 36.6 / 116.9 / 21.3 with cross-entropy loss and 38.7 /129.1 / 22.4 with CIDEr optimization on MSCOCO Karpathytest split, which surpass the results of other state-of-the-artmethods.