# Towards Effective and Efficient Adversarial Defense with Diffusion Models for Robust Visual Tracking

arXiv: https://arxiv.org/abs/2506.00325 · HTML: https://mlsanity.com/paper/2506.00325/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 31 May 2025.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 12 of 12)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [Adversarially Robust Distillation](https://mlsanity.com/paper/1905.09747/) (2019)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [Erasing, Transforming, and Noising Defense Network for Occluded Person Re-Identification](https://mlsanity.com/paper/2307.07187/) (2023)
- [Stimulating Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling](https://mlsanity.com/paper/2307.03992/) (2023)
- [Multi-view Information Integration and Propagation for Occluded Person Re-identification](https://mlsanity.com/paper/2311.03828/) (2023)
- [SiamMask: A Framework for Fast Online Object Tracking and Segmentation](https://mlsanity.com/paper/2207.02088/) (2022)

## Built on by (most cited first, 2 of 2)

- [ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuning](https://mlsanity.com/paper/2510.15331/) (2025)
- [Breaking the Illusion: Consensus-Based Generative Mitigation of Adversarial Illusions in Multi-Modal Embeddings](https://mlsanity.com/paper/2511.21893/) (2025)

## Abstract (arXiv)

Although deep learning-based visual tracking methods have made significant progress, they exhibit vulnerabilities when facing carefully designed adversarial attacks, which can lead to a sharp decline in tracking performance. To address this issue, this paper proposes for the first time a novel adversarial defense method based on denoise diffusion probabilistic models, termed DiffDf, aimed at effectively improving the robustness of existing visual tracking methods against adversarial attacks. DiffDf establishes a multi-scale defense mechanism by combining pixel-level reconstruction loss, semantic consistency loss, and structural similarity loss, effectively suppressing adversarial perturbations through a gradual denoising process. Extensive experimental results on several mainstream datasets show that the DiffDf method demonstrates excellent generalization performance for trackers with different architectures, significantly improving various evaluation metrics while achieving real-time inference speeds of over 30 FPS, showcasing outstanding defense performance and efficiency. Codes are available at https://github.com/pgao-lab/DiffDf.