# ICECAP: Information Concentrated Entity-aware Image Captioning

arXiv: https://arxiv.org/abs/2108.02050 · HTML: https://mlsanity.com/paper/2108.02050/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning](https://mlsanity.com/paper/1612.01887/) (2016)
- [Dual Attention Networks for Multimodal Reasoning and Matching](https://mlsanity.com/paper/1611.00471/) (2016)
- [Good News, Everyone! Context driven entity-aware captioning for news images](https://mlsanity.com/paper/1904.01475/) (2019)
- [Transform and Tell: Entity-Aware News Image Captioning](https://mlsanity.com/paper/2004.08070/) (2020)
- [Instance-aware Image and Sentence Matching with Selective Multimodal LSTM](https://mlsanity.com/paper/1611.05588/) (2016)
- [Entity-aware Image Caption Generation](https://mlsanity.com/paper/1804.07889/) (2018)
- [BreakingNews: Article Annotation by Image and Text Processing](https://mlsanity.com/paper/1603.07141/) (2016)
- [Informative Image Captioning with External Sources of Information](https://mlsanity.com/paper/1906.08876/) (2019)

## Built on by (most cited first, 11 of 11)

- [InfoMetIC: An Informative Metric for Reference-free Image Caption Evaluation](https://mlsanity.com/paper/2305.06002/) (2023)
- [Journalistic Guidelines Aware News Image Captioning](https://mlsanity.com/paper/2109.02865/) (2021)
- [Focus! Relevant and Sufficient Context Selection for News Image Captioning](https://mlsanity.com/paper/2212.00843/) (2022)
- [Generating image captions with external encyclopedic knowledge](https://mlsanity.com/paper/2210.04806/) (2022)
- [Explore and Tell: Embodied Visual Captioning in 3D Environments](https://mlsanity.com/paper/2308.10447/) (2023)
- [Knowledge Completes the Vision: A Multimodal Entity-aware Retrieval-Augmented Generation Framework for News Image Captioning](https://mlsanity.com/paper/2511.21002/) (2025)
- [EAMA : Entity-Aware Multimodal Alignment Based Approach for News Image Captioning](https://mlsanity.com/paper/2402.19404/) (2024)
- [Rule-driven News Captioning](https://mlsanity.com/paper/2403.05101/) (2024)
- [How to Understand Named Entities: Using Common Sense for News Captioning](https://mlsanity.com/paper/2403.06520/) (2024)
- [TROPE: TRaining-Free Object-Part Enhancement for Seamlessly Improving Fine-Grained Zero-Shot Image Captioning](https://mlsanity.com/paper/2409.19960/) (2024)
- [ReCap: Event-Aware Image Captioning with Article Retrieval and Semantic Gaussian Normalization](https://mlsanity.com/paper/2509.01259/) (2025)

## Abstract (arXiv)

Most current image captioning systems focus on describing general image content, and lack background knowledge to deeply understand the image, such as exact named entities or concrete events. In this work, we focus on the entity-aware news image captioning task which aims to generate informative captions by leveraging the associated news articles to provide background knowledge about the target image. However, due to the length of news articles, previous works only employ news articles at the coarse article or sentence level, which are not fine-grained enough to refine relevant events and choose named entities accurately. To overcome these limitations, we propose an Information Concentrated Entity-aware news image CAPtioning (ICECAP) model, which progressively concentrates on relevant textual information within the corresponding news article from the sentence level to the word level. Our model first creates coarse concentration on relevant sentences using a cross-modality retrieval model and then generates captions by further concentrating on relevant words within the sentences. Extensive experiments on both BreakingNews and GoodNews datasets demonstrate the effectiveness of our proposed method, which outperforms other state-of-the-arts. The code of ICECAP is publicly available at https://github.com/HAWLYQ/ICECAP.