Data to 14 Oct 2026

Meshed-Memory Transformer for Image Captioning

arXiv 1912.08226 ↗First posted 17 Dec 2019cs.CV cs.CLcited by 343

Not on the map. Why?

Cited by 24 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Transformer-based architectures represent the state of the art in sequence modeling tasks like machine translation and language understanding. Their applicability to multi-modal contexts like image captioning, however, is still largely under-explored. With the aim of filling this gap, we present M2 - a Meshed Transformer with Memory for Image Captioning. The architecture improves both the image encoding and the language generation steps: it learns a multi-level representation of the relationships between image regions integrating learned a priori knowledge, and uses a mesh-like connectivity at decoding stage to exploit low- and high-level features. Experimentally, we investigate the performance of the M2 Transformer and different fully-attentive models in comparison with recurrent ones. When tested on COCO, our proposal achieves a new state of the art in single-model and ensemble configurations on the "Karpathy" test split and on the online test server. We also assess its performances when describing objects unseen in the training set. Trained models and code for reproducing the experiments are publicly available at: https://github.com/aimagelab/meshed-memory-transformer.

Key facts. Cited by 343 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 83; 2022: 101; 2023: 74; 2024: 43; 2025: 34; 2026: 8.

343 papers.

Sort:

By area:

2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
2021A Survey of Transformerscited by 401
2022Multimodal Learning with Transformers: A Surveycited by 400
2022Transformers in Medical Imaging: A Surveycited by 245
2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 191
2022Cross-modal Memory Networks for Radiology Report Generationcited by 185
2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 176
2022Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learningcited by 159

38 more

2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
2023Interactive and Explainable Region-guided Radiology Report Generationcited by 140
2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 114
2021Causal Attention for Vision-Language Taskscited by 111
2021ArtEmis: Affective Language for Visual Artcited by 104
2023METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokenscited by 101
2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 95
2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 93
2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 78
2023Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluationcited by 78
2022Competence-based Multimodal Curriculum Learning for Medical Report Generationcited by 76
2021Cross Modal Retrieval with Querybank Normalisationcited by 72
2021Dual-Level Collaborative Transformer for Image Captioningcited by 71
2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
2022NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesiscited by 71
2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
2022AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generationcited by 65
2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
2023KiUT: Knowledge-injected U-Transformer for Radiology Report Generationcited by 62
2022NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Taskscited by 60
2022Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformercited by 59
2022Cross-modal Prototype Driven Network for Radiology Report Generationcited by 59
2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
2021TransZero: Attribute-guided Transformer for Zero-Shot Learningcited by 54
2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
2022NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Modelscited by 50
2021Describing and Localizing Multiple Changes with Transformerscited by 47
2021Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generationcited by 47
2024Polos: Multimodal Metric Learning from Human Feedback for Image Captioningcited by 46
2022A Medical Semantic-Assisted Transformer for Radiographic Report Generationcited by 44
2021Progressive Transformer-Based Generation of Radiology Reportscited by 43
2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43

50 most cited of 343 All 343 (JSON)