mlsanityData to 14 Oct 2026

TEAM-Net: Multi-modal Learning for Video Action Recognition with Partial Decoding

arXiv 2110.08814 ↗First posted 17 Oct 2021cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Most of existing video action recognition models ingest raw RGB frames. However, the raw video stream requires enormous storage and contains significant temporal redundancy. Video compression (e.g., H.264, MPEG-4) reduces superfluous information by representing the raw video stream using the concept of Group of Pictures (GOP). Each GOP is composed of the first I-frame (aka RGB image) followed by a number of P-frames, represented by motion vectors and residuals, which can be regarded and used as pre-extracted features. In this work, we 1) introduce sampling the input for the network from partially decoded videos based on the GOP-level, and 2) propose a plug-and-play mulTi-modal lEArning Module (TEAM) for training the network using information from I-frames and P-frames in an end-to-end manner. We demonstrate the superior performance of TEAM-Net compared to the baseline using RGB only. TEAM-Net also achieves the state-of-the-art performance in the area of video action recognition with partial decoding. Code is provided at https://github.com/villawang/TEAM-Net.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 17 Oct 2021.

Builds on

33 of 48 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  7. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  10. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  11. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  12. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
21 more
  1. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  2. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  3. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 387
  4. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  5. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  6. 2019Use What You Have: Video Retrieval Using Representations From Collaborative Expertscited by 214
  7. 2020TEA: Temporal Excitation and Aggregation for Action Recognitioncited by 165
  8. 2019STM: SpatioTemporal and Motion Encoding for Action Recognitioncited by 142
  9. 2019MMTM: Multimodal Transfer Module for CNN Fusioncited by 129
  10. 2017Compressed Video Action Recognitioncited by 106
  11. 20182D/3D Pose Estimation and Action Recognition using Multitask Deep Learningcited by 103
  12. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  13. 2021Involution: Inverting the Inherence of Convolution for Visual Recognitioncited by 99
  14. 2020A Comprehensive Study of Deep Video Action Recognitioncited by 88
  15. 2021ACTION-Net: Multipath Excitation for Action Recognitioncited by 63
  16. 2018Seeing Voices and Hearing Faces: Cross-modal biometric matchingcited by 60
  17. 2016Real-time Action Recognition with Enhanced Motion Vector CNNscited by 51
  18. 2019DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognitioncited by 36
  19. 2018Multi-Fiber Networks for Video Recognitioncited by 34
  20. 2019Mimic The Raw Domain: Accelerating Action Recognition in the Compressed Domaincited by 8
  21. 2020CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognitioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2026CoPE-VideoLM: Leveraging Codec Primitives For Efficient Video Language Modelingcited by 11
  2. 2026AdaCodec: A Predictive Visual Code for Video MLLMscited by 7
  3. 2025Towards Scalable Modeling of Compressed Videos for Efficient Action Recognitioncited by 2
  4. 2024Advancing Compressed Video Action Recognition through Progressive Knowledge Distillationcited by 1
  5. 2026RESUME: Recurrent State Updates from Motion and Residual Signals for Efficient Video Language Modelingcited by 0