mlsanityData to 7 Oct 2026

Compositional Generalization in Image Captioning

arXiv 1909.04402 ↗First posted 10 Sep 2019cs.LG cs.CL cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Image captioning models are usually evaluated on their ability to describe a held-out set of images, not on their ability to generalize to unseen concepts. We study the problem of compositional generalization, which measures how well a model composes unseen combinations of concepts when describing images. State-of-the-art image captioning models show poor generalization performance on this task. We propose a multi-task model to address the poor performance, that combines caption generation and image--sentence ranking, and uses a decoding mechanism that re-ranks the captions according their similarity to the image. This model is substantially better at generalizing to unseen combinations of concepts compared to state-of-the-art captioning models.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 4; 2023: 10; 2024: 5; 2025: 1.

23 papers.

Sort:

By area:

  1. 2022Does CLIP Bind Concepts? Probing Compositionality in Large Image Modelscited by 101
  2. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  3. 2022Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learningcited by 59
  4. 2023Neural-Logic Human-Object Interaction Detectioncited by 25
  5. 2023Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Groundingcited by 19
  6. 2021Removing Word-Level Spurious Alignment between Images and Pseudo-Captions in Unsupervised Image Captioningcited by 14
  7. 2024Compositional Risk Minimizationcited by 13
  8. 2021The Role of Syntactic Planning in Compositional Image Captioningcited by 12
  9. 2023A Survey on Compositional Generalization in Applicationscited by 12
  10. 2021Compositional Generalization in Multilingual Semantic Parsing over Wikidatacited by 9
  11. 2022Deep soccer captioning with transformer: dataset, semantics-related losses, and multi-level evaluationcited by 4
  12. 2023SETI: Systematicity Evaluation of Textual Inferencecited by 4
11 more
  1. 2023A Large-Scale Multilingual Study of Visual Constraints on Linguistic Selection of Descriptionscited by 2
  2. 2023MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisitioncited by 2
  3. 2024Resilience through Scene Context in Visual Referring Expression Generationcited by 2
  4. 2023Does Vision Accelerate Hierarchical Generalization in Neural Language Learners?cited by 1
  5. 2023Learning to generate and corr- uh I mean repair language in real-timecited by 1
  6. 2024Sequential Compositional Generalization in Multimodal Modelscited by 1
  7. 2023KENGIC: KEyword-driven and N-Gram Graph based Image Captioningcited by 0
  8. 2023Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encodercited by 0
  9. 2024Attention Based Simple Primitives for Open World Compositional Zero-Shot Learningcited by 0
  10. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 0
  11. 2025Long Story Short: Disentangling Compositionality and Long-Caption Understanding in Contrastive VLMscited by 0