mlsanityData to 7 Oct 2026

Video Swin Transformer

arXiv 2106.13230 ↗First posted 24 Jun 2021cs.CV cs.AI cs.LGcited by 1,090

Not on the map. Why?

Cited by 35 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks. These video models are all built on Transformer layers that globally connect patches across the spatial and temporal dimensions. In this paper, we instead advocate an inductive bias of locality in video Transformers, which leads to a better speed-accuracy trade-off compared to previous approaches which compute self-attention globally even with spatial-temporal factorization. The locality of the proposed video architecture is realized by adapting the Swin Transformer designed for the image domain, while continuing to leverage the power of pre-trained image models. Our approach achieves state-of-the-art accuracy on a broad range of video recognition benchmarks, including on action recognition (84.9 top-1 accuracy on Kinetics-400 and 86.1 top-1 accuracy on Kinetics-600 with ~20x less pre-training data and ~3x smaller model size) and temporal modeling (69.6 top-1 accuracy on Something-Something v2). The code and models will be made publicly available at https://github.com/SwinTransformer/Video-Swin-Transformer.

Key facts. Cited by 1,090 papers in the map (data through 7 Oct 2026). Builds on 39 of its 43 references that are in the map. First posted on arXiv 24 Jun 2021.

Builds on

39 of 43 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2016Densely Connected Convolutional Networkscited by 6,691
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2014Going Deeper with Convolutionscited by 4,231
  11. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  12. 2017Non-local Neural Networkscited by 2,290
27 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  2. 2017The Kinetics Human Action Video Datasetcited by 1,894
  3. 2018SlowFast Networks for Video Recognitioncited by 1,862
  4. 2021ViViT: A Video Vision Transformercited by 1,610
  5. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  6. 2017Random Erasing Data Augmentationcited by 1,235
  7. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  8. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  9. 2016Deep Networks with Stochastic Depthcited by 973
  10. 2021Multiscale Vision Transformerscited by 896
  11. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  12. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  13. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  14. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  15. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  16. 2017Relation Networks for Object Detectioncited by 305
  17. 2019Video Classification with Channel-Separated Convolutional Networkscited by 290
  18. 2021Video Transformer Networkcited by 271
  19. 2019Local Relation Networks for Image Recognitioncited by 210
  20. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  21. 2020TEA: Temporal Excitation and Aggregation for Action Recognitioncited by 165
  22. 2020Temporal Pyramid Network for Action Recognitioncited by 163
  23. 2020Disentangled Non-Local Neural Networkscited by 132
  24. 2021VidTr: Video Transformer Without Convolutionscited by 112
  25. 2019Video Modeling with Correlation Networkscited by 73
  26. 2020MotionSqueeze: Neural Motion Feature Learning for Video Understandingcited by 72
  27. 2019More Is Less: Learning Efficient Video Representations by Big-Little Network and Depthwise Temporal Aggregationcited by 64

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 43; 2022: 244; 2023: 287; 2024: 198; 2025: 195; 2026: 123.

1,090 papers.

Sort:

By area:

  1. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  2. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  3. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  4. 2022Token Merging: Your ViT But Fastercited by 995
  5. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  6. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  7. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  8. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  9. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  10. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  11. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  12. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
38 more
  1. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  2. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  3. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  4. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  5. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  6. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
  7. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  8. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  9. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  10. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  11. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  12. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  13. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  14. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  15. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 243
  16. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  17. 2022Egocentric Video-Language Pretrainingcited by 231
  18. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  19. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  20. 2022VRT: A Video Restoration Transformercited by 219
  21. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  22. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  23. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  24. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  25. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  26. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
  27. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  28. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  29. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  30. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  31. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  32. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  33. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 170
  34. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  35. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
  36. 2022Human-to-Robot Imitation in the Wildcited by 157
  37. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  38. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153

50 most cited of 1,090 All 1,090 (JSON)