mlsanityData to 7 Oct 2026

Improving Image Captioning with Better Use of Captions

arXiv 2006.11807 ↗First posted 21 Jun 2020cs.CV cs.CLcited by 84

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Image captioning is a multimodal problem that has drawn extensive attention in both the natural language processing and computer vision community. In this paper, we present a novel image captioning architecture to better explore semantics available in captions and leverage that to enhance both image representation and caption generation. Our models first construct caption-guided visual relationship graphs that introduce beneficial inductive bias using weakly supervised multi-instance learning. The representation is then enhanced with neighbouring and contextual nodes with their textual and visual features. During generation, the model further incorporates visual relationships using multi-task learning for jointly predicting word and object/predicate tag sequences. We perform extensive experiments on the MSCOCO dataset, showing that the proposed framework significantly outperforms the baselines, resulting in the state-of-the-art performance under a wide range of evaluation metrics.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 15; 2024: 31; 2025: 25; 2026: 6.

84 papers.

Sort:

By area:

  1. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  2. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  3. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  4. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  5. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  6. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generationcited by 69
  7. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
  8. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 64
  9. 2024The Neglected Tails in Vision-Language Modelscited by 63
  10. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  11. 2023BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Modelscited by 41
  12. 2022Retrieval-Augmented Transformer for Image Captioningcited by 38
38 more
  1. 2024Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matterscited by 38
  2. 2025Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Themcited by 34
  3. 2025Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learningcited by 33
  4. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 31
  5. 2024EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routingcited by 29
  6. 2024Taming Data and Transformers for Audio Generationcited by 28
  7. 2024Playable Game Generationcited by 26
  8. 2024From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalitiescited by 22
  9. 2025YoChameleon: Personalized Vision and Language Generationcited by 20
  10. 2024Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learningcited by 19
  11. 2025UniVid: The Open-Source Unified Video Modelcited by 18
  12. 2023Wired Perspectives: Multi-View Wire Art Embraces Generative AIcited by 16
  13. 2023HiPA: Enabling One-Step Text-to-Image Diffusion Models via High-Frequency-Promoting Adaptationcited by 16
  14. 2024Condition-Aware Neural Network for Controlled Image Generationcited by 16
  15. 2024Federated Learning in Practice: Reflections and Projectionscited by 15
  16. 2022CaMEL: Mean Teacher Learning for Image Captioningcited by 14
  17. 2023Multi-Modality Deep Network for Extreme Learned Image Compressioncited by 14
  18. 2024Urban Scene Diffusion through Semantic Occupancy Mapcited by 14
  19. 2024Can Textual Semantics Mitigate Sounding Object Segmentation Preference?cited by 13
  20. 2024Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Modelscited by 13
  21. 2024Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-trainingcited by 13
  22. 2023Improving Image Captioning via Predicting Structured Conceptscited by 12
  23. 2024Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputscited by 12
  24. 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agentscited by 12
  25. 2023Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Modelscited by 11
  26. 2024Training-free Composite Scene Generation for Layout-to-Image Synthesiscited by 11
  27. 2025DuMo: Dual Encoder Modulation Network for Precise Concept Erasurecited by 11
  28. 2025FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensicscited by 11
  29. 2024Residual Connections Harm Generative Representation Learningcited by 10
  30. 2024Zero-Shot Multi-Object Scene Completioncited by 9
  31. 2025Towards Understanding Visual Grounding in Visual Language Modelscited by 9
  32. 2024Active Generation for Image Classificationcited by 8
  33. 2024DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Modelscited by 7
  34. 2024TIPO: Text to Image with Text Presampling for Prompt Optimizationcited by 6
  35. 2025Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter Itcited by 6
  36. 2026coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generationcited by 6
  37. 2023Memory Triggers: Unveiling Memorization in Text-To-Image Generative Models through Word-Level Duplicationcited by 5
  38. 2023Understanding (Un)Intended Memorization in Text-to-Image Generative Modelscited by 5

50 most cited of 84 All 84 (JSON)