mlsanityData to 7 Oct 2026

Temporal Segment Networks: Towards Good Practices for Deep Action Recognition

arXiv 1608.00859 ↗First posted 2 Aug 2016cs.CVcited by 899

Not on the map. Why?

Cited by 29 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep convolutional networks have achieved great success for visual recognition in still images. However, for action recognition in videos, the advantage over traditional methods is not so evident. This paper aims to discover the principles to design effective ConvNet architectures for action recognition in videos and learn these models given limited training samples. Our first contribution is temporal segment network (TSN), a novel framework for video-based action recognition. which is based on the idea of long-range temporal structure modeling. It combines a sparse temporal sampling strategy and video-level supervision to enable efficient and effective learning using the whole action video. The other contribution is our study on a series of good practices in learning ConvNets on video data with the help of temporal segment network. Our approach obtains the state-the-of-art performance on the datasets of HMDB51 ( 69.4% ) and UCF101 ( 94.2% ). We also visualize the learned ConvNet models, which qualitatively demonstrates the effectiveness of temporal segment network and the proposed good practices.

Key facts. Cited by 899 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 235; 2022: 220; 2023: 165; 2024: 136; 2025: 90; 2026: 53.

899 papers.

Sort:

By area:

  1. 2021ViViT: A Video Vision Transformercited by 1,610
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  4. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  5. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  6. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  7. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  8. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  9. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  10. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  11. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  12. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
38 more
  1. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  2. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  3. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  4. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  5. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  6. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  7. 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 278
  8. 2021Video Transformer Networkcited by 271
  9. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  10. 2021Revisiting Skeleton-based Action Recognitioncited by 256
  11. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  12. 2021Anabranch Network for Camouflaged Object Segmentationcited by 227
  13. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  14. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  15. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  16. 2023Video Understanding with Large Language Models: A Surveycited by 203
  17. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  18. 2021Anticipative Video Transformercited by 197
  19. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  20. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  21. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
  22. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  23. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  24. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  25. 2021End-to-end Temporal Action Detection with Transformercited by 150
  26. 2021Towards Long-Form Video Understandingcited by 146
  27. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 146
  28. 2021Taming Visually Guided Sound Generationcited by 136
  29. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  30. 2022Revealing Single Frame Bias for Video-and-Language Learningcited by 131
  31. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 116
  32. 2022UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormercited by 116
  33. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  34. 2021VidTr: Video Transformer Without Convolutionscited by 112
  35. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 111
  36. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  37. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  38. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98

50 most cited of 899 All 899 (JSON)