# Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations

arXiv: https://arxiv.org/abs/2110.01823 · HTML: https://mlsanity.com/paper/2110.01823/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 30 of 37)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [Generative Adversarial Perturbations](https://mlsanity.com/paper/1712.02328/) (2017)
- [Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors](https://mlsanity.com/paper/1807.07978/) (2018)
- [Enhancing Adversarial Example Transferability with an Intermediate Level Attack](https://mlsanity.com/paper/1907.10823/) (2019)
- [Cross-Domain Transferability of Adversarial Perturbations](https://mlsanity.com/paper/1905.11736/) (2019)
- [Black-box Adversarial Attacks on Video Recognition Models](https://mlsanity.com/paper/1904.05181/) (2019)
- [Sparse Adversarial Perturbations for Videos](https://mlsanity.com/paper/1803.02536/) (2018)
- [NAG: Network for Adversary Generation](https://mlsanity.com/paper/1712.03390/) (2017)
- [Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation](https://mlsanity.com/paper/1810.05162/) (2018)
- [Heuristic Black-box Adversarial Attacks on Video Recognition Models](https://mlsanity.com/paper/1911.09449/) (2019)
- [Over-the-Air Adversarial Flickering Attacks against Video Recognition Networks](https://mlsanity.com/paper/2002.05123/) (2020)
- [Adversarial Perturbations Against Real-Time Video Classification Systems](https://mlsanity.com/paper/1807.00458/) (2018)
- [Spatio-Temporal Representation Factorization for Video-based Person Re-Identification](https://mlsanity.com/paper/2107.11878/) (2021)
- [Sparse Black-box Video Attack with Reinforcement Learning](https://mlsanity.com/paper/2001.03754/) (2020)
- [GAT: Generative Adversarial Training for Adversarial Example Detection and Robust Classification](https://mlsanity.com/paper/1905.11475/) (2019)
- [Appending Adversarial Frames for Universal Video Attack](https://mlsanity.com/paper/1912.04538/) (2019)

## Built on by (most cited first, 28 of 28)

- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [Efficient Decision-based Black-box Patch Attacks on Video Recognition](https://mlsanity.com/paper/2303.11917/) (2023)
- [StyleFool: Fooling Video Classification Systems via Style Transfer](https://mlsanity.com/paper/2203.16000/) (2022)
- [Cross-Domain Video Anomaly Detection without Target Domain Adaptation](https://mlsanity.com/paper/2212.07010/) (2022)
- [Efficient Robustness Assessment via Adversarial Spatial-Temporal Focus on Videos](https://mlsanity.com/paper/2301.00896/) (2023)
- [Ensemble-based Blackbox Attacks on Dense Prediction](https://mlsanity.com/paper/2303.14304/) (2023)
- [Blackbox Attacks via Surrogate Ensemble Search](https://mlsanity.com/paper/2208.03610/) (2022)
- [GAMA: Generative Adversarial Multi-Object Scene Attacks](https://mlsanity.com/paper/2209.09502/) (2022)
- [ADC: Adversarial attacks against object Detection that evade Context consistency checks](https://mlsanity.com/paper/2110.12321/) (2021)
- [Exploiting Vision Encoder Vulnerabilities for Universal Adversarial Perturbations on Large Vision-Language Models](https://mlsanity.com/paper/2412.08108/) (2024)
- [Adversarial Machine Learning Attacks Against Video Anomaly Detection Systems](https://mlsanity.com/paper/2204.03141/) (2022)
- [Adversarial Attacks on Video Object Segmentation with Hard Region Discovery](https://mlsanity.com/paper/2309.13857/) (2023)
- [Breaking Temporal Consistency: Generating Video Universal Adversarial Perturbations Using Image Models](https://mlsanity.com/paper/2311.10366/) (2023)
- [Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack](https://mlsanity.com/paper/2211.11312/) (2022)
- [Leveraging Local Patch Differences in Multi-Object Scenes for Generative Adversarial Attacks](https://mlsanity.com/paper/2209.09883/) (2022)
- [Multimodal Attack Detection for Action Recognition Models](https://mlsanity.com/paper/2404.10790/) (2024)
- [VideoPure: Diffusion-based Adversarial Purification for Video Recognition](https://mlsanity.com/paper/2501.14999/) (2025)
- [Adversarial Attacks on Event-Based Pedestrian Detectors: A Physical Approach](https://mlsanity.com/paper/2503.00377/) (2025)
- [RoVISQ: Reduction of Video Service Quality via Adversarial Attacks on Deep Learning-based Video Compression](https://mlsanity.com/paper/2203.10183/) (2022)
- [Improve Video Representation with Temporal Adversarial Augmentation](https://mlsanity.com/paper/2304.14601/) (2023)
- [Physics-Based Adversarial Attack on Near-Infrared Human Detector for Nighttime Surveillance Camera Systems](https://mlsanity.com/paper/2412.13709/) (2024)
- [LocalStyleFool: Regional Video Style Transfer Attack Using Segment Anything Model](https://mlsanity.com/paper/2403.11656/) (2024)
- [FrameShield: Adversarially Robust Video Anomaly Detection](https://mlsanity.com/paper/2510.21532/) (2025)
- [SNEAK: Synonymous Sentences-Aware Adversarial Attack on Natural Language Video Localization](https://mlsanity.com/paper/2112.04154/) (2021)
- [Query Efficient Cross-Dataset Transferable Black-Box Attack on Action Recognition](https://mlsanity.com/paper/2211.13171/) (2022)
- [Rigid Body Adversarial Attacks](https://mlsanity.com/paper/2502.05669/) (2025)
- [Robustness Evaluation for Video Models with Reinforcement Learning](https://mlsanity.com/paper/2506.05431/) (2025)
- [FeatureFool: Zero-Query Fooling of Video Models via Feature Map](https://mlsanity.com/paper/2510.18362/) (2025)

## Abstract (arXiv)

When compared to the image classification models, black-box adversarial attacks against video classification models have been largely understudied. This could be possible because, with video, the temporal dimension poses significant additional challenges in gradient estimation. Query-efficient black-box attacks rely on effectively estimated gradients towards maximizing the probability of misclassifying the target video. In this work, we demonstrate that such effective gradients can be searched for by parameterizing the temporal structure of the search space with geometric transformations. Specifically, we design a novel iterative algorithm Geometric TRAnsformed Perturbations (GEO-TRAP), for attacking video classification models. GEO-TRAP employs standard geometric transformation operations to reduce the search space for effective gradients into searching for a small group of parameters that define these operations. This group of parameters describes the geometric progression of gradients, resulting in a reduced and structured search space. Our algorithm inherently leads to successful perturbations with surprisingly few queries. For example, adversarial examples generated from GEO-TRAP have better attack success rates with ~73.55% fewer queries compared to the state-of-the-art method for video adversarial attacks on the widely used Jester dataset. Overall, our algorithm exposes vulnerabilities of diverse video classification models and achieves new state-of-the-art results under black-box settings on two large datasets. Code is available here: https://github.com/sli057/Geo-TRAP