# Towards Evaluating the Robustness of Visual State Space Models

arXiv: https://arxiv.org/abs/2406.09407 · HTML: https://mlsanity.com/paper/2406.09407/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2024.

## Built on by (most cited first, 5 of 5)

- [Mamba in Vision: A Comprehensive Survey of Techniques and Applications](https://mlsanity.com/paper/2410.03105/) (2024)
- [Concealed Adversarial attacks on neural networks for sequential data](https://mlsanity.com/paper/2502.20948/) (2025)
- [CLIMP: Contrastive Language-Image Mamba Pretraining](https://mlsanity.com/paper/2601.06891/) (2026)
- [Efficient Multi-Source Knowledge Transfer by Model Merging](https://mlsanity.com/paper/2508.19353/) (2025)
- [Time-Scaling State-Space Models for Dense Video Captioning](https://mlsanity.com/paper/2509.03426/) (2025)

## Abstract (arXiv)

Vision State Space Models (VSSMs), a novel architecture that combines the strengths of recurrent neural networks and latent variable models, have demonstrated remarkable performance in visual perception tasks by efficiently capturing long-range dependencies and modeling complex visual dynamics. However, their robustness under natural and adversarial perturbations remains a critical concern. In this work, we present a comprehensive evaluation of VSSMs' robustness under various perturbation scenarios, including occlusions, image structure, common corruptions, and adversarial attacks, and compare their performance to well-established architectures such as transformers and Convolutional Neural Networks. Furthermore, we investigate the resilience of VSSMs to object-background compositional changes on sophisticated benchmarks designed to test model performance in complex visual scenes. We also assess their robustness on object detection and segmentation tasks using corrupted datasets that mimic real-world scenarios. To gain a deeper understanding of VSSMs' adversarial robustness, we conduct a frequency-based analysis of adversarial attacks, evaluating their performance against low-frequency and high-frequency perturbations. Our findings highlight the strengths and limitations of VSSMs in handling complex visual corruptions, offering valuable insights for future research. Our code and models will be available at https://github.com/HashmatShadab/MambaRobustness.