mlsanityData to 14 Oct 2026

CPTR: Full Transformer Network for Image Captioning

arXiv 2101.10804 ↗First posted 26 Jan 2021cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

In this paper, we consider the image captioning task from a new sequence-to-sequence prediction perspective and propose CaPtion TransformeR (CPTR) which takes the sequentialized raw images as the input to Transformer. Compared to the "CNN+Transformer" design paradigm, our model can model global context at every encoder layer from the beginning and is totally convolution-free. Extensive experiments demonstrate the effectiveness of the proposed model and we surpass the conventional "CNN+Transformer" methods on the MSCOCO dataset. Besides, we provide detailed visualizations of the self-attention between patches in the encoder and the "words-to-patches" attention in the decoder thanks to the full Transformer architecture.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 17 of its 19 references that are in the map. First posted on arXiv 26 Jan 2021.

Builds on

17 of 19 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  7. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  8. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  9. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  10. 2020Pre-Trained Image Processing Transformercited by 893
  11. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  12. 2016Self-critical Sequence Training for Image Captioningcited by 547
5 more
  1. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 255
  2. 2018Exploring Visual Relationship for Image Captioningcited by 215
  3. 2019Image Captioning: Transforming Objects into Wordscited by 171
  4. 2016Boosting Image Captioning with Attributescited by 76
  5. 2019Reflective Decoding Network for Image Captioningcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 13; 2023: 15; 2024: 10; 2025: 3; 2026: 2.

55 papers.

Sort:

By area:

  1. 2021Multiscale Vision Transformerscited by 897
  2. 2021ImageNet-21K Pretraining for the Massescited by 613
  3. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  4. 2021Remote Sensing Image Change Detection with Transformerscited by 306
  5. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
  6. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  7. 2022Causal Transformer for Estimating Counterfactual Outcomescited by 123
  8. 2021Beyond Self-attention: External Attention using Two Linear Layers for Visual Taskscited by 109
  9. 2022RelTR: Relation Transformer for Scene Graph Generationcited by 96
  10. 2023DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignmentcited by 90
  11. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  12. 2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 78
38 more
  1. 2021Audio Captioning Transformercited by 55
  2. 2023PRIOR: Prototype Representation Joint Learning from Medical Images and Reportscited by 55
  3. 2021A Survey on Multimodal Disinformation Detectioncited by 47
  4. 2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
  5. 2023Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioningcited by 39
  6. 2022Retrieval-Augmented Transformer for Image Captioningcited by 38
  7. 2023Retrieval-augmented Image Captioningcited by 29
  8. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  9. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  10. 2024Generative Multi-modal Models are Good Class-Incremental Learnerscited by 22
  11. 2023Exploring Efficient Few-shot Adaptation for Vision Transformerscited by 16
  12. 2022CaMEL: Mean Teacher Learning for Image Captioningcited by 14
  13. 2022SYMBA: Symbolic Computation of Squared Amplitudes in High Energy Physics with Machine Learningcited by 13
  14. 2023Adjacent-Level Feature Cross-Fusion With 3-D CNN for Remote Sensing Image Change Detectioncited by 13
  15. 2024Image Captioning in news report scenariocited by 12
  16. 2024Towards Retrieval-Augmented Architectures for Image Captioningcited by 7
  17. 2022Using Human Perception to Regularize Transfer Learningcited by 6
  18. 2023Embodied Agents for Efficient Exploration and Smart Scene Descriptioncited by 6
  19. 2023SEM-POS: Grammatically and Semantically Correct Video Captioningcited by 6
  20. 2024PaveCap: The First Multimodal Framework for Comprehensive Pavement Condition Assessment with Dense Captioning and PCI Estimationcited by 6
  21. 2022Automatic Generation of Product-Image Sequence in E-commercecited by 5
  22. 2022BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrievalcited by 4
  23. 2024MvKeTR: Chest CT Report Generation with Multi-View Perception and Knowledge Enhancementcited by 4
  24. 2022Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detectioncited by 3
  25. 2023GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-trainingcited by 3
  26. 2023Radiology Report Generation Using Transformers Conditioned with Non-imaging Datacited by 3
  27. 2024CAPEEN: Image Captioning with Early Exits and Knowledge Distillationcited by 3
  28. 2021Bangla Image Caption Generation through CNN-Transformer based Encoder-Decoder Networkcited by 2
  29. 2022Are metrics measuring what they should? An evaluation of image captioning task metricscited by 2
  30. 2025A Deep Learning Model of Mental Rotation Informed by Interactive VR Experimentscited by 2
  31. 2023A Comprehensive Analysis of Real-World Image Captioning and Scene Identificationcited by 1
  32. 2023Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generationcited by 1
  33. 2024Rule-driven News Captioningcited by 1
  34. 2021Partially-Supervised Novel Object Captioning Leveraging Context from Paired Datacited by 0
  35. 2022Describing image focused in cognitive and visual details for visually impaired people: An approach to generating inclusive paragraphscited by 0
  36. 2023Adaptively Clustering Neighbor Elements for Image-Text Generationcited by 0
  37. 2024CarbonNet: How Computer Vision Plays a Role in Climate Change? Application: Learning Geomechanics from Subsurface Geometry of CCS to Mitigate Global Warmingcited by 0
  38. 2024Figuring out Figures: Using Textual References to Caption Scientific Figurescited by 0

50 most cited of 55 All 55 (JSON)