# TEINet: Towards an Efficient Architecture for Video Recognition

arXiv: https://arxiv.org/abs/1911.09435 · HTML: https://mlsanity.com/paper/1911.09435/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 30 of 92)

- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [UniFormer: Unifying Convolution and Self-attention for Visual Recognition](https://mlsanity.com/paper/2201.09450/) (2022)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [VidTr: Video Transformer Without Convolutions](https://mlsanity.com/paper/2104.11746/) (2021)
- [Spatiotemporal Inconsistency Learning for DeepFake Video Detection](https://mlsanity.com/paper/2109.01860/) (2021)
- [Revisiting Classifier: Transferring Vision-Language Models for Video Recognition](https://mlsanity.com/paper/2207.01297/) (2022)
- [ACTION-Net: Multipath Excitation for Action Recognition](https://mlsanity.com/paper/2103.07372/) (2021)
- [Efficient Video Transformers with Spatial-Temporal Token Selection](https://mlsanity.com/paper/2111.11591/) (2021)
- [Continuous Sign Language Recognition with Correlation Network](https://mlsanity.com/paper/2303.03202/) (2023)
- [MGSampler: An Explainable Sampling Strategy for Video Action Recognition](https://mlsanity.com/paper/2104.09952/) (2021)
- [Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2301.00182/) (2022)
- [AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition](https://mlsanity.com/paper/2102.05775/) (2021)
- [AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders](https://mlsanity.com/paper/2211.09120/) (2022)
- [Long-Short Temporal Contrastive Learning of Video Transformers](https://mlsanity.com/paper/2106.09212/) (2021)
- [MGMAE: Motion Guided Masking for Video Masked Autoencoding](https://mlsanity.com/paper/2308.10794/) (2023)
- [VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling](https://mlsanity.com/paper/2406.04321/) (2024)
- [AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognition](https://mlsanity.com/paper/2112.14238/) (2021)
- [DCAN: Improving Temporal Action Detection via Dual Context Aggregation](https://mlsanity.com/paper/2112.03612/) (2021)
- [Video Test-Time Adaptation for Action Recognition](https://mlsanity.com/paper/2211.15393/) (2022)
- [Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognition](https://mlsanity.com/paper/2102.07092/) (2021)
- [BasicTAD: an Astounding RGB-Only Baseline for Temporal Action Detection](https://mlsanity.com/paper/2205.02717/) (2022)
- [CT-Net: Channel Tensorization Network for Video Classification](https://mlsanity.com/paper/2106.01603/) (2021)
- [DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection](https://mlsanity.com/paper/2306.00863/) (2023)
- [OCSampler: Compressing Videos to One Clip with Single-step Sampling](https://mlsanity.com/paper/2201.04388/) (2022)
- [MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection](https://mlsanity.com/paper/2211.10996/) (2022)
- [MorphMLP: An Efficient MLP-Like Backbone for Spatial-Temporal Representation Learning](https://mlsanity.com/paper/2111.12527/) (2021)
- [Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection](https://mlsanity.com/paper/2403.10261/) (2024)

## Abstract (arXiv)

Efficiency is an important issue in designing video architectures for action recognition. 3D CNNs have witnessed remarkable progress in action recognition from videos. However, compared with their 2D counterparts, 3D convolutions often introduce a large amount of parameters and cause high computational cost. To relieve this problem, we propose an efficient temporal module, termed as Temporal Enhancement-and-Interaction (TEI Module), which could be plugged into the existing 2D CNNs (denoted by TEINet). The TEI module presents a different paradigm to learn temporal features by decoupling the modeling of channel correlation and temporal interaction. First, it contains a Motion Enhanced Module (MEM) which is to enhance the motion-related features while suppress irrelevant information (e.g., background). Then, it introduces a Temporal Interaction Module (TIM) which supplements the temporal contextual information in a channel-wise manner. This two-stage modeling scheme is not only able to capture temporal structure flexibly and effectively, but also efficient for model inference. We conduct extensive experiments to verify the effectiveness of TEINet on several benchmarks (e.g., Something-Something V1&V2, Kinetics, UCF101 and HMDB51). Our proposed TEINet can achieve a good recognition accuracy on these datasets but still preserve a high efficiency.