# Temporal Distinct Representation Learning for Action Recognition

arXiv: https://arxiv.org/abs/2007.07626 · HTML: https://mlsanity.com/paper/2007.07626/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2020.

## Built on by (most cited first, 13 of 13)

- [MGSampler: An Explainable Sampling Strategy for Video Action Recognition](https://mlsanity.com/paper/2104.09952/) (2021)
- [Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.04337/) (2021)
- [CT-Net: Channel Tensorization Network for Video Classification](https://mlsanity.com/paper/2106.01603/) (2021)
- [Motion Sensitive Contrastive Learning for Self-supervised Video Representation](https://mlsanity.com/paper/2208.06105/) (2022)
- [Towards Long Video Understanding via Fine-detailed Video Story Generation](https://mlsanity.com/paper/2412.06182/) (2024)
- [Video Is Graph: Structured Graph Module for Video Action Recognition](https://mlsanity.com/paper/2110.05904/) (2021)
- [Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos](https://mlsanity.com/paper/2212.13495/) (2022)
- [Learning Comprehensive Motion Representation for Action Recognition](https://mlsanity.com/paper/2103.12278/) (2021)
- [Temporal Transformer Networks with Self-Supervision for Action Recognition](https://mlsanity.com/paper/2112.07338/) (2021)
- [Exploring the Diversity and Invariance in Yourself for Visual Pre-Training Task](https://mlsanity.com/paper/2106.00537/) (2021)
- [TSI: Temporal Saliency Integration for Video Action Recognition](https://mlsanity.com/paper/2106.01088/) (2021)
- [ST-ABN: Visual Explanation Taking into Account Spatio-temporal Information for Video Recognition](https://mlsanity.com/paper/2110.15574/) (2021)
- [Context-Aware Network Based on Multi-scale Spatio-temporal Attention for Action Recognition in Videos](https://mlsanity.com/paper/2512.18750/) (2025)

## Abstract (arXiv)

Motivated by the previous success of Two-Dimensional Convolutional Neural Network (2D CNN) on image recognition, researchers endeavor to leverage it to characterize videos. However, one limitation of applying 2D CNN to analyze videos is that different frames of a video share the same 2D CNN kernels, which may result in repeated and redundant information utilization, especially in the spatial semantics extraction process, hence neglecting the critical variations among frames. In this paper, we attempt to tackle this issue through two ways. 1) Design a sequential channel filtering mechanism, i.e., Progressive Enhancement Module (PEM), to excite the discriminative channels of features from different frames step by step, and thus avoid repeated information extraction. 2) Create a Temporal Diversity Loss (TD Loss) to force the kernels to concentrate on and capture the variations among frames rather than the image regions with similar appearance. Our method is evaluated on benchmark temporal reasoning datasets Something-Something V1 and V2, and it achieves visible improvements over the best competitor by 2.4% and 1.3%, respectively. Besides, performance improvements over the 2D-CNN-based state-of-the-arts on the large-scale dataset Kinetics are also witnessed.