# Speeding Up Action Recognition Using Dynamic Accumulation of Residuals in Compressed Domain

arXiv: https://arxiv.org/abs/2209.14757 · HTML: https://mlsanity.com/paper/2209.14757/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition](https://mlsanity.com/paper/1901.03460/) (2019)
- [Fast Object Detection in Compressed Video](https://mlsanity.com/paper/1811.11057/) (2018)
- [Non-Linear Temporal Subspace Representations for Activity Recognition](https://mlsanity.com/paper/1803.11064/) (2018)
- [Pooled Motion Features for First-Person Videos](https://mlsanity.com/paper/1412.6505/) (2014)
- [Convolutional Long Short-Term Memory Networks for Recognizing First Person Interactions](https://mlsanity.com/paper/1709.06495/) (2017)
- [Mimic The Raw Domain: Accelerating Action Recognition in the Compressed Domain](https://mlsanity.com/paper/1911.08206/) (2019)
- [Real-time Online Multi-Object Tracking in Compressed Domain](https://mlsanity.com/paper/2204.02081/) (2022)
- [Three-Stream Fusion Network for First-Person Interaction Recognition](https://mlsanity.com/paper/2002.08219/) (2020)

## Abstract (arXiv)

With the widespread use of installed cameras, video-based monitoring approaches have seized considerable attention for different purposes like assisted living. Temporal redundancy and the sheer size of raw videos are the two most common problematic issues related to video processing algorithms. Most of the existing methods mainly focused on increasing accuracy by exploring consecutive frames, which is laborious and cannot be considered for real-time applications. Since videos are mostly stored and transmitted in compressed format, these kinds of videos are available on many devices. Compressed videos contain a multitude of beneficial information, such as motion vectors and quantized coefficients. Proper use of this available information can greatly improve the video understanding methods' performance. This paper presents an approach for using residual data, available in compressed videos directly, which can be obtained by a light partially decoding procedure. In addition, a method for accumulating similar residuals is proposed, which dramatically reduces the number of processed frames for action recognition. Applying neural networks exclusively for accumulated residuals in the compressed domain accelerates performance, while the classification results are highly competitive with raw video approaches.