mlsanityData to 14 Oct 2026

Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioning

arXiv 2208.06551 ↗First posted 13 Aug 2022cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

We introduce a method called the Expansion mechanism that processes the input unconstrained by the number of elements in the sequence. By doing so, the model can learn more effectively compared to traditional attention-based approaches. To support this claim, we design a novel architecture ExpansionNet v2 that achieved strong results on the MS COCO 2014 Image Captioning challenge and the State of the Art in its respective category, with a score of 143.7 CIDErD in the offline test split, 140.8 CIDErD in the online evaluation server and 72.9 AllCIDEr on the nocaps validation set. Additionally, we introduce an End to End training algorithm up to 2.8 times faster than established alternatives. Source code available at: https://github.com/jchenghu/ExpansionNet_v2

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 45 of its 56 references that are in the map. First posted on arXiv 13 Aug 2022.

Builds on

45 of 56 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  9. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  11. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
33 more
  1. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  2. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  3. 2020On Layer Normalization in the Transformer Architecturecited by 922
  4. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  5. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  6. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  7. 2016Self-critical Sequence Training for Image Captioningcited by 547
  8. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  9. 2020Hopfield Networks is All You Needcited by 475
  10. 2018nocaps: novel object captioning at scalecited by 444
  11. 2021FNet: Mixing Tokens with Fourier Transformscited by 347
  12. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  13. 2019Attention on Attention for Image Captioningcited by 289
  14. 2018Bilinear Attention Networkscited by 280
  15. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 255
  16. 2018Exploring Visual Relationship for Image Captioningcited by 215
  17. 2020Synthesizer: Rethinking Self-Attention in Transformer Modelscited by 205
  18. 2020X-Linear Attention Networks for Image Captioningcited by 198
  19. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  20. 2019Image Captioning: Transforming Objects into Wordscited by 171
  21. 2016Guided Open Vocabulary Image Captioning with Constrained Beam Searchcited by 105
  22. 2019Star-Transformercited by 94
  23. 2019Augmenting Self-attention with Persistent Memorycited by 93
  24. 2021Dual-Level Collaborative Transformer for Image Captioningcited by 71
  25. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  26. 2020Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Networkcited by 48
  27. 2020Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translationcited by 43
  28. 2020Hard-Coded Gaussian Attention for Neural Machine Translationcited by 42
  29. 2022End-to-End Transformer Based Model for Image Captioningcited by 38
  30. 2020Show, Recall, and Tell: Image Captioning with Recall Mechanismcited by 27
  31. 2019Modeling Recurrence for Transformercited by 26
  32. 2020A Better Variant of Self-Critical Sequence Trainingcited by 23
  33. 2022Exploring the sequence length bottleneck in the Transformer for Image Captioningcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 7; 2024: 4; 2025: 7.

18 papers.

Sort:

By area:

  1. 2023Tuning computer vision models with task rewardscited by 37
  2. 2023A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answeringcited by 14
  3. 2023A Study of Autoregressive Decoders for Multi-Tasking in Computer Visioncited by 10
  4. 2025Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captionscited by 10
  5. 2024LEMoN: Label Error Detection using Multimodal Neighborscited by 6
  6. 2024AICAttack: Adversarial Image Captioning Attack with Attention-Based Optimizationcited by 5
  7. 2025DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cyclescited by 5
  8. 2023A Survey on Image-text Multimodal Modelscited by 4
  9. 2023Learning Human-Human Interactions in Images from Weak Textual Supervisioncited by 3
  10. 2025GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoningcited by 3
  11. 2023A request for clarity over the End of Sequence token in the Self-Critical Sequence Trainingcited by 2
  12. 2024Bidirectional Awareness Induction in Autoregressive Seq2Seq Modelscited by 2
6 more
  1. 2024Shifted Window Fourier Transform And Retention For Image Captioningcited by 2
  2. 2023Heterogeneous Encoders Scaling In The Transformer For Neural Machine Translationcited by 1
  3. 2025A parameter-efficient three-branch architecture for multimodal misinformation detection with limited annotationscited by 1
  4. 2025Visual moral inference and communicationcited by 0
  5. 2025SEMT: Static-Expansion-Mesh Transformer Network Architecture for Remote Sensing Image Captioningcited by 0
  6. 2025Diffusion Is Your Friend in Show, Suggest and Tellcited by 0