mlsanityData to 14 Oct 2026

Robustness Evaluation for Video Models with Reinforcement Learning

arXiv 2506.05431 ↗First posted 5 Jun 2025cs.CV cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Evaluating the robustness of Video classification models is very challenging, specifically when compared to image-based models. With their increased temporal dimension, there is a significant increase in complexity and computational cost. One of the key challenges is to keep the perturbations to a minimum to induce misclassification. In this work, we propose a multi-agent reinforcement learning approach (spatial and temporal) that cooperatively learns to identify the given video's sensitive spatial and temporal regions. The agents consider temporal coherence in generating fine perturbations, leading to a more effective and visually imperceptible attack. Our method outperforms the state-of-the-art solutions on the Lp metric and the average queries. Our method enables custom distortion types, making the robustness evaluation more relevant to the use case. We extensively evaluate 4 popular models for video action recognition on two popular datasets, HMDB-51 and UCF-101.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 40 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

25 of 40 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018SlowFast Networks for Video Recognitioncited by 1,867
  3. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  4. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  5. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  6. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 446
  7. 2018Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priorscited by 146
  8. 2017Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Rewardcited by 105
  9. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  10. 2019Black-box Adversarial Attacks on Video Recognition Modelscited by 58
  11. 2018Sparse Adversarial Perturbations for Videoscited by 57
  12. 2019Heuristic Black-box Adversarial Attacks on Video Recognition Modelscited by 37
13 more
  1. 2018Adversarial Perturbations Against Real-Time Video Classification Systemscited by 32
  2. 2021Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformationscited by 28
  3. 2020Sparse Black-box Video Attack with Reinforcement Learningcited by 24
  4. 2021Boosting the Transferability of Video Adversarial Examples via Temporal Translationcited by 21
  5. 2023Efficient Robustness Assessment via Adversarial Spatial-Temporal Focus on Videoscited by 20
  6. 2019Appending Adversarial Frames for Universal Video Attackcited by 17
  7. 2020Motion-Excited Sampler: Video Adversarial Attack with Sparked Priorcited by 17
  8. 2020Just One Moment: Structural Vulnerability of Deep Action Recognition against One Frame Attackcited by 12
  9. 2021Reinforcement Learning Based Sparse Black-box Adversarial Attack on Video Recognition Modelscited by 12
  10. 2025Cross-Modal Transferable Image-to-Video Attack on Video Quality Metricscited by 9
  11. 2023Benchmark Generation Framework with Customizable Distortions for Image Classifier Robustnesscited by 7
  12. 2024Robustness and Visual Explanation for Black Box Image, Video, and ECG Signal Classification with Reinforcement Learningcited by 4
  13. 2025Reinforcement Learning Platform for Adversarial Black-box Attacks with Custom Distortion Filterscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Principled Representation Learning from Videos for Reinforcement Learning
  2. 2026Learning Latent Action World Models In The Wild
  3. 2026Test-time Reinforcement Learning for Anomalous Video Understanding
  4. 2024Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations
  5. 2025Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Space
  6. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
  7. 2026From Pixels to Temporal Correlations: Learning Informative Representations for Reinforcement Learning Pre-training
  8. 2026World Models as Group Actions