# Performance Evaluation of Action Recognition Models on Low Quality Videos

arXiv: https://arxiv.org/abs/2204.09166 · HTML: https://mlsanity.com/paper/2204.09166/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [Moments in Time Dataset: one million videos for event understanding](https://mlsanity.com/paper/1801.03150/) (2018)
- [Why do deep convolutional networks generalize so poorly to small image transformations?](https://mlsanity.com/paper/1805.12177/) (2018)
- [Understanding How Image Quality Affects Deep Neural Networks](https://mlsanity.com/paper/1604.04004/) (2016)
- [A Comprehensive Study of Deep Video Action Recognition](https://mlsanity.com/paper/2012.06567/) (2020)
- [Video Transformers: A Survey](https://mlsanity.com/paper/2201.05991/) (2022)
- [Studying Very Low Resolution Recognition Using Deep Networks](https://mlsanity.com/paper/1601.04153/) (2016)
- [PyTorchVideo: A Deep Learning Library for Video Understanding](https://mlsanity.com/paper/2111.09887/) (2021)
- [How Image Degradations Affect Deep CNN-based Face Recognition?](https://mlsanity.com/paper/1608.05246/) (2016)
- [Dual Directed Capsule Network for Very Low Resolution Image Recognition](https://mlsanity.com/paper/1908.10027/) (2019)
- [Strengths and Weaknesses of Deep Learning Models for Face Recognition Against Image Degradations](https://mlsanity.com/paper/1710.01494/) (2017)
- [TinyVIRAT: Low-resolution Video Action Recognition](https://mlsanity.com/paper/2007.07355/) (2020)
- [Video Action Understanding](https://mlsanity.com/paper/2010.06647/) (2020)
- [Convolutional Low-Resolution Fine-Grained Classification](https://mlsanity.com/paper/1703.05393/) (2017)
- [Minimal Images in Deep Neural Networks: Fragile Object Recognition in Natural Images](https://mlsanity.com/paper/1902.03227/) (2019)
- [Enhance Visual Recognition under Adverse Conditions via Deep Networks](https://mlsanity.com/paper/1712.07732/) (2017)
- [Synthesis-based Robust Low Resolution Face Recognition](https://mlsanity.com/paper/1707.02733/) (2017)

## Built on by (most cited first, 3 of 3)

- [Deep Video Codec Control for Vision Models](https://mlsanity.com/paper/2308.16215/) (2023)
- [A Perspective on Deep Vision Performance with Standard Image and Video Codecs](https://mlsanity.com/paper/2404.12330/) (2024)
- [Multi-model learning by sequential reading of untrimmed videos for action recognition](https://mlsanity.com/paper/2401.14675/) (2024)

## Abstract (arXiv)

In the design of action recognition models, the quality of videos is an important issue; however, the trade-off between the quality and performance is often ignored. In general, action recognition models are trained on high-quality videos, hence it is not known how the model performance degrades when tested on low-quality videos, and how much the quality of training videos affects the performance. The issue of video quality is important, however, it has not been studied so far. The goal of this study is to show the trade-off between the performance and the quality of training and test videos by quantitative performance evaluation of several action recognition models for transcoded videos in different qualities. First, we show how the video quality affects the performance of pre-trained models. We transcode the original validation videos of Kinetics400 by changing quality control parameters of JPEG (compression strength) and H.264/AVC (CRF). Then we use the transcoded videos to validate the pre-trained models. Second, we show how the models perform when trained on transcoded videos. We transcode the original training videos of Kinetics400 by changing the quality parameters of JPEG and H.264/AVC. Then we train the models on the transcoded training videos and validate them with the original and transcoded validation videos. Experimental results with JPEG transcoding show that there is no severe performance degradation (up to -1.5%) for compression strength smaller than 70 where no quality degradation is visually observed, and for larger than 80 the performance degrades linearly with respect to the quality index. Experiments with H.264/AVC transcoding show that there is no significant performance loss (up to -1%) with CRF30 while the total size of video files is reduced to 30%.