# Long-term Temporal Convolutions for Action Recognition

arXiv: https://arxiv.org/abs/1604.04494 · HTML: https://mlsanity.com/paper/1604.04494/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2016.

## Built on by (most cited first, 30 of 84)

- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [Temporal-wise Attention Spiking Neural Networks for Event Streams Classification](https://mlsanity.com/paper/2107.11711/) (2021)
- [InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction](https://mlsanity.com/paper/2209.12354/) (2022)
- [Object-Region Video Transformers](https://mlsanity.com/paper/2110.06915/) (2021)
- [Temporal Query Networks for Fine-grained Video Understanding](https://mlsanity.com/paper/2104.09496/) (2021)
- [HierVL: Learning Hierarchical Video-Language Embeddings](https://mlsanity.com/paper/2301.02311/) (2023)
- [Simple Hardware-Efficient Long Convolutions for Sequence Modeling](https://mlsanity.com/paper/2302.06646/) (2023)
- [A CLIP-Hitchhiker's Guide to Long Video Retrieval](https://mlsanity.com/paper/2205.08508/) (2022)
- [Home Action Genome: Cooperative Compositional Action Understanding](https://mlsanity.com/paper/2105.05226/) (2021)
- [MGSampler: An Explainable Sampling Strategy for Video Action Recognition](https://mlsanity.com/paper/2104.09952/) (2021)
- [Local-Global Context Aware Transformer for Language-Guided Video Segmentation](https://mlsanity.com/paper/2203.09773/) (2022)
- [SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos](https://mlsanity.com/paper/2104.11452/) (2021)
- [MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain](https://mlsanity.com/paper/2209.08691/) (2022)
- [Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning](https://mlsanity.com/paper/2103.10211/) (2021)
- [Coarse-Fine Networks for Temporal Activity Detection in Videos](https://mlsanity.com/paper/2103.01302/) (2021)
- [Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foreground](https://mlsanity.com/paper/2211.12883/) (2022)
- [Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition](https://mlsanity.com/paper/2107.00606/) (2021)
- [D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2111.07774/) (2021)
- [RMS-Net: Regression and Masking for Soccer Event Spotting](https://mlsanity.com/paper/2102.07624/) (2021)
- [Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognition](https://mlsanity.com/paper/2209.01425/) (2022)
- [End-to-End Compressed Video Representation Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2203.15336/) (2022)
- [Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognition](https://mlsanity.com/paper/2101.06636/) (2021)
- [Deep Unsupervised Key Frame Extraction for Efficient Video Classification](https://mlsanity.com/paper/2211.06742/) (2022)
- [TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction](https://mlsanity.com/paper/2109.00953/) (2021)
- [Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition](https://mlsanity.com/paper/2110.14994/) (2021)
- [Continual Transformers: Redundancy-Free Attention for Online Inference](https://mlsanity.com/paper/2201.06268/) (2022)
- [HERMES: temporal-coHERent long-forM understanding with Episodes and Semantics](https://mlsanity.com/paper/2408.17443/) (2024)
- [3D Skeleton-based Few-shot Action Recognition with JEANIE is not so Na"ive](https://mlsanity.com/paper/2112.12668/) (2021)

## Abstract (arXiv)

Typical human actions last several seconds and exhibit characteristic spatio-temporal structure. Recent methods attempt to capture this structure and learn action representations with convolutional neural networks. Such representations, however, are typically learned at the level of a few video frames failing to model actions at their full temporal extent. In this work we learn video representations using neural networks with long-term temporal convolutions (LTC). We demonstrate that LTC-CNN models with increased temporal extents improve the accuracy of action recognition. We also study the impact of different low-level representations, such as raw values of video pixels and optical flow vector fields and demonstrate the importance of high-quality optical flow estimation for learning accurate action models. We report state-of-the-art results on two challenging benchmarks for human action recognition UCF101 (92.7%) and HMDB51 (67.2%).