# LocalStyleFool: Regional Video Style Transfer Attack Using Segment Anything Model

arXiv: https://arxiv.org/abs/2403.11656 · HTML: https://mlsanity.com/paper/2403.11656/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Track Anything: Segment Anything Meets Videos](https://mlsanity.com/paper/2304.11968/) (2023)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples](https://mlsanity.com/paper/1811.12673/) (2018)
- [Black-box Adversarial Attacks on Video Recognition Models](https://mlsanity.com/paper/1904.05181/) (2019)
- [Sparse Adversarial Perturbations for Videos](https://mlsanity.com/paper/1803.02536/) (2018)
- [Heuristic Black-box Adversarial Attacks on Video Recognition Models](https://mlsanity.com/paper/1911.09449/) (2019)
- [Over-the-Air Adversarial Flickering Attacks against Video Recognition Networks](https://mlsanity.com/paper/2002.05123/) (2020)
- [Adversarial Perturbations Against Real-Time Video Classification Systems](https://mlsanity.com/paper/1807.00458/) (2018)
- [Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations](https://mlsanity.com/paper/2110.01823/) (2021)
- [BASAR:Black-box Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2103.05266/) (2021)
- [Sparse Black-box Video Attack with Reinforcement Learning](https://mlsanity.com/paper/2001.03754/) (2020)
- [Efficient Decision-based Black-box Patch Attacks on Video Recognition](https://mlsanity.com/paper/2303.11917/) (2023)
- [StyleFool: Fooling Video Classification Systems via Style Transfer](https://mlsanity.com/paper/2203.16000/) (2022)

## Built on by (most cited first, 2 of 2)

- [Threats, Attacks, and Defenses in Machine Unlearning: A Survey](https://mlsanity.com/paper/2403.13682/) (2024)
- [Query-Efficient Video Adversarial Attack with Stylized Logo on Service Computing](https://mlsanity.com/paper/2408.12099/) (2024)

## Abstract (arXiv)

Previous work has shown that well-crafted adversarial perturbations can threaten the security of video recognition systems. Attackers can invade such models with a low query budget when the perturbations are semantic-invariant, such as StyleFool. Despite the query efficiency, the naturalness of the minutia areas still requires amelioration, since StyleFool leverages style transfer to all pixels in each frame. To close the gap, we propose LocalStyleFool, an improved black-box video adversarial attack that superimposes regional style-transfer-based perturbations on videos. Benefiting from the popularity and scalably usability of Segment Anything Model (SAM), we first extract different regions according to semantic information and then track them through the video stream to maintain the temporal consistency. Then, we add style-transfer-based perturbations to several regions selected based on the associative criterion of transfer-based gradient information and regional area. Perturbation fine adjustment is followed to make stylized videos adversarial. We demonstrate that LocalStyleFool can improve both intra-frame and inter-frame naturalness through a human-assessed survey, while maintaining competitive fooling rate and query efficiency. Successful experiments on the high-resolution dataset also showcase that scrupulous segmentation of SAM helps to improve the scalability of adversarial attacks under high-resolution data.