mlsanityData to 7 Oct 2026

TSM: Temporal Shift Module for Efficient Video Understanding

arXiv 1811.08383 ↗First posted 20 Nov 2018cs.CVcited by 852

Not on the map. Why?

Cited by 20 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The explosive growth in video streaming gives rise to challenges on performing video understanding at high accuracy and low computation cost. Conventional 2D CNNs are computationally cheap but cannot capture temporal relationships; 3D CNN based methods can achieve good performance but are computationally intensive, making it expensive to deploy. In this paper, we propose a generic and effective Temporal Shift Module (TSM) that enjoys both high efficiency and high performance. Specifically, it can achieve the performance of 3D CNN but maintain 2D CNN's complexity. TSM shifts part of the channels along the temporal dimension; thus facilitate information exchanged among neighboring frames. It can be inserted into 2D CNNs to achieve temporal modeling at zero computation and zero parameters. We also extended TSM to online setting, which enables real-time low-latency online video recognition and video object detection. TSM is accurate and efficient: it ranks the first place on the Something-Something leaderboard upon publication; on Jetson Nano and Galaxy Note8, it achieves a low latency of 13ms and 35ms for online video recognition. The code is available at: https://github.com/mit-han-lab/temporal-shift-module.

Key facts. Cited by 852 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 182; 2022: 186; 2023: 150; 2024: 125; 2025: 117; 2026: 92.

852 papers.

Sort:

By area:

  1. 2021ViViT: A Video Vision Transformercited by 1,610
  2. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  3. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  4. 2021Video Swin Transformercited by 1,090
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2021Multiscale Vision Transformerscited by 896
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  8. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  9. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  10. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  11. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  12. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
38 more
  1. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  2. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  3. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  4. 2021Video Transformer Networkcited by 271
  5. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  6. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  7. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
  8. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  9. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  10. 2023A Survey on Video Diffusion Modelscited by 203
  11. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  12. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  13. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
  14. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  15. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  16. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  17. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  18. 2022Human-to-Robot Imitation in the Wildcited by 157
  19. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  20. 2022Multiview Transformers for Video Recognitioncited by 148
  21. 2023AIM: Adapting Image Models for Efficient Video Action Recognitioncited by 145
  22. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137
  23. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  24. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  25. 2022Vision-Centric BEV Perception: A Surveycited by 126
  26. 2021ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysiscited by 124
  27. 2021Vision Transformer for Small-Size Datasetscited by 122
  28. 2021PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformercited by 118
  29. 2021A Survey of Visual Transformerscited by 117
  30. 2022UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormercited by 116
  31. 2021VidTr: Video Transformer Without Convolutionscited by 112
  32. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 111
  33. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 111
  34. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
  35. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  36. 2022TS2-Net: Token Shift and Selection Transformer for Text-Video Retrievalcited by 104
  37. 2021AS-MLP: An Axial Shifted MLP Architecture for Visioncited by 101
  38. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101

50 most cited of 852 All 852 (JSON)