# Temporal Pyramid Pooling Based Convolutional Neural Networks for Action Recognition

arXiv: https://arxiv.org/abs/1503.01224 · HTML: https://mlsanity.com/paper/1503.01224/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2015.

## Built on by (most cited first, 15 of 15)

- [Content and Style Aware Generation of Text-line Images for Handwriting Recognition](https://mlsanity.com/paper/2204.05539/) (2022)
- [Coarse to Fine Multi-Resolution Temporal Convolutional Network](https://mlsanity.com/paper/2105.10859/) (2021)
- [Motion-Focused Contrastive Learning of Video Representations](https://mlsanity.com/paper/2201.04029/) (2022)
- [C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation](https://mlsanity.com/paper/2212.11078/) (2022)
- [HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Series](https://mlsanity.com/paper/2510.25785/) (2025)
- [Understanding Spatio-Temporal Relations in Human-Object Interaction using Pyramid Graph Convolutional Network](https://mlsanity.com/paper/2410.07912/) (2024)
- [Rethinking HTG Evaluation: Bridging Generation and Recognition](https://mlsanity.com/paper/2409.02683/) (2024)
- [An Identity-Preserved Framework for Human Motion Transfer](https://mlsanity.com/paper/2204.06862/) (2022)
- [Time Scale Network: A Shallow Neural Network For Time Series Data](https://mlsanity.com/paper/2311.06170/) (2023)
- [Literature on Hand GESTURE Recognition using Graph based methods](https://mlsanity.com/paper/2207.00329/) (2022)
- [Visualizing the Passage of Time with Video Temporal Pyramids](https://mlsanity.com/paper/2208.11885/) (2022)
- [3DPyranet Features Fusion for Spatio-temporal Feature Learning](https://mlsanity.com/paper/2504.18977/) (2025)
- [Recognition of Daily Activities through Multi-Modal Deep Learning: A Video, Pose, and Object-Aware Approach for Ambient Assisted Living](https://mlsanity.com/paper/2603.04509/) (2026)
- [Hi-GaTA: Hierarchical Gated Temporal Aggregation Adapter for Surgical Video Report Generation](https://mlsanity.com/paper/2605.11208/) (2026)
- [VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfer](https://mlsanity.com/paper/2605.28229/) (2026)

## Abstract (arXiv)

Encouraged by the success of Convolutional Neural Networks (CNNs) in image classification, recently much effort is spent on applying CNNs to video based action recognition problems. One challenge is that video contains a varying number of frames which is incompatible to the standard input format of CNNs. Existing methods handle this issue either by directly sampling a fixed number of frames or bypassing this issue by introducing a 3D convolutional layer which conducts convolution in spatial-temporal domain. To solve this issue, here we propose a novel network structure which allows an arbitrary number of frames as the network input. The key of our solution is to introduce a module consisting of an encoding layer and a temporal pyramid pooling layer. The encoding layer maps the activation from previous layers to a feature vector suitable for pooling while the temporal pyramid pooling layer converts multiple frame-level activations into a fixed-length video-level representation. In addition, we adopt a feature concatenation layer which combines appearance information and motion information. Compared with the frame sampling strategy, our method avoids the risk of missing any important frames. Compared with the 3D convolutional method which requires a huge video dataset for network training, our model can be learned on a small target dataset because we can leverage the off-the-shelf image-level CNN for model parameter initialization. Experiments on two challenging datasets, Hollywood2 and HMDB51, demonstrate that our method achieves superior performance over state-of-the-art methods while requiring much fewer training data.