mlsanityData to 7 Oct 2026

Entity-aware Image Caption Generation

arXiv 1804.07889 ↗First posted 21 Apr 2018cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

Current image captioning approaches generate descriptions which lack specific information, such as named entities that are involved in the images. In this paper we propose a new task which aims to generate informative image captions, given images and hashtags as input. We propose a simple but effective approach to tackle this problem. We first train a convolutional neural networks - long short term memory networks (CNN-LSTM) model to generate a template caption based on the input image. Then we use a knowledge graph based collective inference algorithm to fill in the template with specific named entities retrieved via the hashtags. Experiments on a new benchmark dataset collected from Flickr show that our model generates news-style image descriptions with much richer information. Our model outperforms unimodal baselines significantly with various evaluation metrics.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 6; 2023: 4; 2024: 8; 2025: 2.

26 papers.

Sort:

By area:

  1. 2022Translation between Molecules and Natural Languagecited by 173
  2. 2022A Decade of Knowledge Graphs in Natural Language Processing: A Surveycited by 40
  3. 2021Boosting Entity-aware Image Captioning with Multi-modal Knowledge Graphcited by 31
  4. 2021Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generationcited by 30
  5. 2021Journalistic Guidelines Aware News Image Captioningcited by 14
  6. 2021ICECAP: Information Concentrated Entity-aware Image Captioningcited by 11
  7. 2022CapOnImage: Context-driven Dense-Captioning on Imagecited by 11
  8. 2023Beyond Generic: Enhancing Image Captioning with Real-World Knowledge using Vision-Language Pre-Training Modelcited by 10
  9. 2021IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry Systemcited by 6
  10. 2021Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interactioncited by 5
  11. 2024Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformationcited by 5
  12. 2024Web-Scale Visual Entity Recognition: An LLM-Driven Data Approachcited by 5
14 more
  1. 2023IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of Imagescited by 4
  2. 2022Generating image captions with external encyclopedic knowledgecited by 3
  3. 2024Enhancing Image Caption Generation Using Reinforcement Learning with Human Feedbackcited by 2
  4. 2024EAMA : Entity-Aware Multimodal Alignment Based Approach for News Image Captioningcited by 1
  5. 2024A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)cited by 1
  6. 2025TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generationcited by 1
  7. 2022Controllable Image Captioningcited by 0
  8. 2022Towards Multimodal Vision-Language Models Generating Non-Generic Textcited by 0
  9. 2023HICL: Hashtag-Driven In-Context Learning for Social Media Natural Language Understandingcited by 0
  10. 2023Video Summarization: Towards Entity-Aware Captionscited by 0
  11. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 0
  12. 2024TROPE: TRaining-Free Object-Part Enhancement for Seamlessly Improving Fine-Grained Zero-Shot Image Captioningcited by 0
  13. 2024A Pattern to Align Them All: Integrating Different Modalities to Define Multi-Modal Entitiescited by 0
  14. 2025Beam-Guided Knowledge Replay for Knowledge-Rich Image Captioning using Vision-Language Modelcited by 0