mlsanityData to 14 Oct 2026

Image Captioning: Transforming Objects into Words

arXiv 1906.05963 ↗First posted 14 Jun 2019cs.CV cs.CLcited by 171

Not on the map. Why?

Abstract (arXiv)

Image captioning models typically follow an encoder-decoder architecture which uses abstract image feature vectors as input to the encoder. One of the most successful algorithms uses feature vectors extracted from the region proposals obtained from an object detector. In this work we introduce the Object Relation Transformer, that builds upon this approach by explicitly incorporating information about the spatial relationship between input detected objects through geometric attention. Quantitative and qualitative results demonstrate the importance of such geometric attention for image captioning, leading to improvements on all common captioning metrics on the MS-COCO dataset.

Key facts. Cited by 171 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 47; 2022: 46; 2023: 34; 2024: 20; 2025: 15; 2026: 9.

171 papers.

Sort:

By area:

  1. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  2. 2022Multimodal Learning with Transformers: A Surveycited by 400
  3. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  4. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  5. 2022Translation between Molecules and Natural Languagecited by 177
  6. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 172
  7. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  8. 2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
  9. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  10. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  11. 2022Text-Only Training for Image Captioning using Noise-Injected CLIPcited by 91
  12. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
38 more
  1. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuningcited by 79
  2. 2023Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluationcited by 78
  3. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  4. 2021Dual-Level Collaborative Transformer for Image Captioningcited by 71
  5. 2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 62
  6. 2021HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learningcited by 57
  7. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  8. 2021CPTR: Full Transformer Network for Image Captioningcited by 55
  9. 2021TransZero: Attribute-guided Transformer for Zero-Shot Learningcited by 54
  10. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Modelcited by 53
  11. 2021Describing and Localizing Multiple Changes with Transformerscited by 47
  12. 2024Polos: Multimodal Metric Learning from Human Feedback for Image Captioningcited by 46
  13. 2023Can We Edit Multimodal Large Language Models?cited by 44
  14. 2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
  15. 2022Modeling Image Composition for Complex Scene Generationcited by 43
  16. 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
  17. 2021ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generationcited by 41
  18. 2022Spatiality-guided Transformer for 3D Dense Captioning on Point Cloudscited by 40
  19. 2023PoseFix: Correcting 3D Human Poses with Natural Languagecited by 40
  20. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  21. 2022End-to-End Transformer Based Model for Image Captioningcited by 38
  22. 2022Retrieval-Augmented Transformer for Image Captioningcited by 38
  23. 2022Semantic-Conditional Diffusion Networks for Image Captioningcited by 38
  24. 2023Relational Context Learning for Human-Object Interaction Detectioncited by 36
  25. 2022Comprehending and Ordering Semantics for Image Captioningcited by 33
  26. 2021Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classificationcited by 31
  27. 2023Retrieval-augmented Image Captioningcited by 29
  28. 2021TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learningcited by 28
  29. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  30. 2021Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioningcited by 27
  31. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  32. 2023Towards Local Visual Modeling for Image Captioningcited by 26
  33. 2022Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioningcited by 25
  34. 2021ReFormer: The Relational Transformer for Image Captioningcited by 23
  35. 2021Exploiting Cross-Modal Prediction and Relation Consistency for Semi-Supervised Image Captioningcited by 23
  36. 2022Noise-aware Learning from Web-crawled Image-Text Data for Image Captioningcited by 22
  37. 2024MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistencycited by 22
  38. 2022Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Taskscited by 21

50 most cited of 171 All 171 (JSON)