mlsanityData to 7 Oct 2026

You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection

arXiv 2106.00666 ↗First posted 1 Jun 2021cs.CV cs.AI cs.LGcited by 170

Not on the map. Why?

Abstract (arXiv)

Can Transformer perform 2D object- and region-level recognition from a pure sequence-to-sequence perspective with minimal knowledge about the 2D spatial structure? To answer this question, we present You Only Look at One Sequence (YOLOS), a series of object detection models based on the vanilla Vision Transformer with the fewest possible modifications, region priors, as well as inductive biases of the target task. We find that YOLOS pre-trained on the mid-sized ImageNet-1k dataset only can already achieve quite competitive performance on the challenging COCO object detection benchmark, e.g., YOLOS-Base directly adopted from BERT-Base architecture can obtain 42.0 box AP on COCO val. We also discuss the impacts as well as limitations of current pre-train schemes and model scaling strategies for Transformer in vision through YOLOS. Code and pre-trained models are available at https://github.com/hustvl/YOLOS.

Key facts. Cited by 170 papers in the map (data through 7 Oct 2026). Builds on 59 of its 88 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

59 of 88 references.

References · 59

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  12. 2020End-to-End Object Detection with Transformerscited by 7,384
38 more
  1. 2017Mask R-CNNcited by 7,259
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2016Layer Normalizationcited by 5,579
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  13. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  14. 2020Longformer: The Long-Document Transformercited by 3,362
  15. 2016Wide Residual Networkscited by 3,057
  16. 2015Fast R-CNNcited by 2,930
  17. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  18. 2018YOLOv3: An Incremental Improvementcited by 2,652
  19. 2017Non-local Neural Networkscited by 2,290
  20. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  21. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  22. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  23. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  24. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  25. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  26. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  27. 2017Random Erasing Data Augmentationcited by 1,235
  28. 2019Objects as Pointscited by 1,172
  29. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  30. 2016Deep Networks with Stochastic Depthcited by 973
  31. 2020Designing Network Design Spacescited by 954
  32. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  33. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  34. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  35. 2018Rethinking ImageNet Pre-trainingcited by 376
  36. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  37. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  38. 2017Relation Networks for Object Detectioncited by 305

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 46; 2023: 48; 2024: 36; 2025: 18; 2026: 13.

170 papers.

Sort:

By area:

  1. 2023Vision Transformers Need Registerscited by 844
  2. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
  3. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  4. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  5. 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 235
  6. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  7. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  8. 2023A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAScited by 149
  9. 2022Multimodal Token Fusion for Vision Transformerscited by 132
  10. 2021A Survey of Visual Transformerscited by 117
  11. 2024Video Diffusion Alignment via Reward Gradientscited by 101
  12. 2022Inception Transformercited by 99
38 more
  1. 2022HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentationcited by 94
  2. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  3. 2021TransMix: Attend to Mix for Vision Transformerscited by 79
  4. 2023HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusioncited by 68
  5. 2022Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstructioncited by 67
  6. 2022Relationformer: A Unified Framework for Image-to-Graph Generationcited by 67
  7. 2022Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imagingcited by 66
  8. 2023FasterViT: Fast Vision Transformers with Hierarchical Attentioncited by 65
  9. 2025Your ViT is Secretly an Image Segmentation Modelcited by 61
  10. 2022Where in the World is this Image? Transformer-based Geo-localization in the Wildcited by 58
  11. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  12. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  13. 2022Temporally Efficient Vision Transformer for Video Instance Segmentationcited by 55
  14. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  15. 2023Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Constructioncited by 50
  16. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  17. 2021ViDT: An Efficient and Effective Fully Transformer-based Object Detectorcited by 45
  18. 2023Less is More: Focus Attention for Efficient DETRcited by 38
  19. 2022Towards Data-Efficient Detection Transformerscited by 34
  20. 2022Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detectioncited by 33
  21. 2022Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformerscited by 33
  22. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
  23. 2023FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoringcited by 30
  24. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  25. 2022EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithmcited by 27
  26. 2022PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimationcited by 27
  27. 2021Towards End-to-End Image Compression and Analysis with Transformerscited by 26
  28. 2021APRIL: Finding the Achilles' Heel on Privacy for Vision Transformerscited by 23
  29. 2022Self-Promoted Supervision for Few-Shot Transformercited by 22
  30. 2023Object Detection with Transformers: A Reviewcited by 22
  31. 2022RNTrajRec: Road Network Enhanced Trajectory Recovery with Spatial-Temporal Transformercited by 21
  32. 2025Spiking Vision Transformer with Saccadic Attentioncited by 21
  33. 2022Bridged Transformer for Vision and Point Cloud 3D Object Detectioncited by 19
  34. 2023SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolutioncited by 18
  35. 2023CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrappingcited by 17
  36. 2022Attention Distillation: self-supervised vision transformer students need more guidancecited by 16
  37. 2022YORO -- Lightweight End to End Visual Groundingcited by 16
  38. 2023Scattering Vision Transformer: Spectral Mixing Matterscited by 16

50 most cited of 170 All 170 (JSON)