# Sequential Deep Trajectory Descriptor for Action Recognition with Three-stream CNN

arXiv: https://arxiv.org/abs/1609.03056 · HTML: https://mlsanity.com/paper/1609.03056/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2016.

## Built on by (most cited first, 5 of 5)

- [Versatile Learned Video Compression](https://mlsanity.com/paper/2111.03386/) (2021)
- [Anomaly Detection using Edge Computing in Video Surveillance System: Review](https://mlsanity.com/paper/2107.02778/) (2021)
- [Video Is Graph: Structured Graph Module for Video Action Recognition](https://mlsanity.com/paper/2110.05904/) (2021)
- [A Multi-Modal Transformer Network for Action Detection](https://mlsanity.com/paper/2305.19624/) (2023)
- [A Novel Deep Hybrid Framework with Ensemble-Based Feature Optimization for Robust Real-Time Human Activity Recognition](https://mlsanity.com/paper/2508.18695/) (2025)

## Abstract (arXiv)

Learning the spatial-temporal representation of motion information is crucial to human action recognition. Nevertheless, most of the existing features or descriptors cannot capture motion information effectively, especially for long-term motion. To address this problem, this paper proposes a long-term motion descriptor called sequential Deep Trajectory Descriptor (sDTD). Specifically, we project dense trajectories into two-dimensional planes, and subsequently a CNN-RNN network is employed to learn an effective representation for long-term motion. Unlike the popular two-stream ConvNets, the sDTD stream is introduced into a three-stream framework so as to identify actions from a video sequence. Consequently, this three-stream framework can simultaneously capture static spatial features, short-term motion and long-term motion in the video. Extensive experiments were conducted on three challenging datasets: KTH, HMDB51 and UCF101. Experimental results show that our method achieves state-of-the-art performance on the KTH and UCF101 datasets, and is comparable to the state-of-the-art methods on the HMDB51 dataset.