# Benchmarking the Robustness of Spatial-Temporal Models Against Corruptions

arXiv: https://arxiv.org/abs/2110.06513 · HTML: https://mlsanity.com/paper/2110.06513/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Certified Adversarial Robustness with Additive Noise](https://mlsanity.com/paper/1809.03113/) (2018)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [Adversarial Examples Are a Natural Consequence of Test Error in Noise](https://mlsanity.com/paper/1901.10513/) (2019)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation](https://mlsanity.com/paper/1906.02611/) (2019)
- [Only Time Can Tell: Discovering Temporal Data for Temporal Modeling](https://mlsanity.com/paper/1907.08340/) (2019)
- [More Is Less: Learning Efficient Video Representations by Big-Little Network and Depthwise Temporal Aggregation](https://mlsanity.com/paper/1912.00869/) (2019)
- [Do Image Classifiers Generalize Across Time?](https://mlsanity.com/paper/1906.02168/) (2019)
- [Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition](https://mlsanity.com/paper/2010.11757/) (2020)

## Built on by (most cited first, 30 of 34)

- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives](https://mlsanity.com/paper/2501.04003/) (2025)
- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)
- [RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions](https://mlsanity.com/paper/2310.15171/) (2023)
- [Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Trigger](https://mlsanity.com/paper/2302.14677/) (2023)
- [Video Test-Time Adaptation for Action Recognition](https://mlsanity.com/paper/2211.15393/) (2022)
- [Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models](https://mlsanity.com/paper/2306.02080/) (2023)
- [Robustness of SAM: Segment Anything Under Corruptions and Beyond](https://mlsanity.com/paper/2306.07713/) (2023)
- [Large-scale Robustness Analysis of Video Action Recognition Models](https://mlsanity.com/paper/2207.01398/) (2022)
- [RobustMQ: Benchmarking Robustness of Quantized Models](https://mlsanity.com/paper/2308.02350/) (2023)
- [A Coding Framework and Benchmark towards Low-Bitrate Video Understanding](https://mlsanity.com/paper/2202.02813/) (2022)
- [Referring Atomic Video Action Recognition](https://mlsanity.com/paper/2407.01872/) (2024)
- [Temporal Coherent Test-Time Optimization for Robust Video Classification](https://mlsanity.com/paper/2302.14309/) (2023)
- [The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation](https://mlsanity.com/paper/2307.15061/) (2023)
- [Benchmarking the Robustness of Temporal Action Detection Models Against Temporal Corruptions](https://mlsanity.com/paper/2403.20254/) (2024)
- [SMC++: Masked Learning of Unsupervised Video Semantic Compression](https://mlsanity.com/paper/2406.04765/) (2024)
- [Robust and Transferable Backdoor Attacks Against Deep Image Compression With Selective Frequency Prior](https://mlsanity.com/paper/2412.01646/) (2024)
- [Recur, Attend or Convolve? On Whether Temporal Modeling Matters for Cross-Domain Robustness in Action Recognition](https://mlsanity.com/paper/2112.12175/) (2021)
- [Adversarial Backdoor Attack by Naturalistic Data Poisoning on Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2306.15755/) (2023)
- [ModSelect: Automatic Modality Selection for Synthetic-to-Real Domain Generalization](https://mlsanity.com/paper/2208.09414/) (2022)
- [Uncovering the Hidden Dynamics of Video Self-supervised Learning under Distribution Shifts](https://mlsanity.com/paper/2306.02014/) (2023)
- [Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions](https://mlsanity.com/paper/2411.14865/) (2024)
- [Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation](https://mlsanity.com/paper/2506.12481/) (2025)
- [Assessing Visually-Continuous Corruption Robustness of Neural Networks Relative to Human Performance](https://mlsanity.com/paper/2402.19401/) (2024)
- [RobustFormer: Noise-Robust Pre-training for images and videos](https://mlsanity.com/paper/2411.13040/) (2024)
- [Investigating and Enhancing the Robustness of Large Multimodal Models Against Temporal Inconsistency](https://mlsanity.com/paper/2505.14405/) (2025)
- [Adversarial Augmentation Training Makes Action Recognition Models More Robust to Realistic Video Distribution Shifts](https://mlsanity.com/paper/2401.11406/) (2024)
- [Adaptive Sensitivity Analysis for Robust Augmentation against Natural Corruptions in Image Segmentation](https://mlsanity.com/paper/2406.01425/) (2024)
- [SHARDeg: A Benchmark for Skeletal Human Action Recognition in Degraded Scenarios](https://mlsanity.com/paper/2505.18048/) (2025)
- [RobustA: Robust Anomaly Detection in Multimodal Data](https://mlsanity.com/paper/2511.07276/) (2025)

## Abstract (arXiv)

The state-of-the-art deep neural networks are vulnerable to common corruptions (e.g., input data degradations, distortions, and disturbances caused by weather changes, system error, and processing). While much progress has been made in analyzing and improving the robustness of models in image understanding, the robustness in video understanding is largely unexplored. In this paper, we establish a corruption robustness benchmark, Mini Kinetics-C and Mini SSV2-C, which considers temporal corruptions beyond spatial corruptions in images. We make the first attempt to conduct an exhaustive study on the corruption robustness of established CNN-based and Transformer-based spatial-temporal models. The study provides some guidance on robust model design and training: Transformer-based model performs better than CNN-based models on corruption robustness; the generalization ability of spatial-temporal models implies robustness against temporal corruptions; model corruption robustness (especially robustness in the temporal domain) enhances with computational cost and model capacity, which may contradict the current trend of improving the computational efficiency of models. Moreover, we find the robustness intervention for image-related tasks (e.g., training models with noise) may not work for spatial-temporal models.