mlsanityData to 7 Oct 2026

Exploring Adversarial Robustness of Deep State Space Models

arXiv 2406.05532 ↗First posted 8 Jun 2024cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Deep State Space Models (SSMs) have proven effective in numerous task scenarios but face significant security challenges due to Adversarial Perturbations (APs) in real-world deployments. Adversarial Training (AT) is a mainstream approach to enhancing Adversarial Robustness (AR) and has been validated on various traditional DNN architectures. However, its effectiveness in improving the AR of SSMs remains unclear. While many enhancements in SSM components, such as integrating Attention mechanisms and expanding to data-dependent SSM parameterizations, have brought significant gains in Standard Training (ST) settings, their potential benefits in AT remain unexplored. To investigate this, we evaluate existing structural variants of SSMs with AT to assess their AR performance. We observe that pure SSM structures struggle to benefit from AT, whereas incorporating Attention yields a markedly better trade-off between robustness and generalization for SSMs in AT compared to other components. Nonetheless, the integration of Attention also leads to Robust Overfitting (RO) issues. To understand these phenomena, we empirically and theoretically analyze the output error of SSMs under AP. We find that fixed-parameterized SSMs have output error bounds strictly related to their parameters, limiting their AT benefits, while input-dependent SSMs may face the problem of error explosion. Furthermore, we show that the Attention component effectively scales the output error of SSMs during training, enabling them to benefit more from AT, but at the cost of introducing RO due to its high model complexity. Inspired by this, we propose a simple and effective Adaptive Scaling (AdS) mechanism that brings AT performance close to Attention-integrated SSMs without introducing the issue of RO. Our code is available at https://github.com/Biqing-Qi/Exploring-Adversarial-Robustness-of-Deep-State-Space-Models.git.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 44 of its 54 references that are in the map. First posted on arXiv 8 Jun 2024.

Builds on

44 of 54 references.

References · 44

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  6. 2016Adversarial examples in the physical worldcited by 1,762
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  9. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  10. 2024VMamba: Visual State Space Modelcited by 955
  11. 2016Adversarial Machine Learning at Scalecited by 934
  12. 2018Robustness May Be at Odds with Accuracycited by 873
32 more
  1. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  2. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  3. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  4. 2019Adversarial Training for Free!cited by 638
  5. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  6. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  7. 2020Overfitting in adversarially robust deep learningcited by 504
  8. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  9. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  10. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  11. 2018Adversarially Robust Generalization Requires More Datacited by 318
  12. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  13. 2022Long Range Language Modeling via Gated State Spacescited by 265
  14. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  15. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  16. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  17. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
  18. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  19. 2022Mega: Moving Average Equipped Gated Attentioncited by 178
  20. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  21. 2018Towards the first adversarially robust neural network model on MNISTcited by 104
  22. 2019Metric Learning for Adversarial Robustnesscited by 77
  23. 2021Exploring Memorization in Adversarial Trainingcited by 55
  24. 2022S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spacescited by 41
  25. 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memorycited by 38
  26. 2019Theoretical evidence for adversarial robustness through randomizationcited by 33
  27. 2020Towards Understanding Fast Adversarial Trainingcited by 32
  28. 2018Theoretical Analysis of Adversarial Learning: A Minimax Approachcited by 31
  29. 2020Uncovering the Connections Between Adversarial Transferability and Knowledge Transferabilitycited by 20
  30. 2023Sparse Modular Activation for Efficient Sequence Modelingcited by 20
  31. 2024Understanding Robustness of Visual State Space Models for Image Classificationcited by 12
  32. 2024Investigating Deep Watermark Security: An Adversarial Transferability Perspectivecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  2. 2024BadVim: Unveiling Backdoor Threats in Visual State Space Modelcited by 1
  3. 2026Adversarial Robustness of Deep State Space Models for Forecastingcited by 1
  4. 2026S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLPcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Evaluating the Robustness of Visual State Space Models
  2. 2024Explaining the role of Intrinsic Dimensionality in Adversarial Training
  3. 2026What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models
  4. 2024Adversarial Feature Alignment: Balancing Robustness and Accuracy in Deep Learning via Adversarial Training
  5. 2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectives
  6. 2025Navigating the Trade-off: A Synthesis of Defensive Strategies for Zero-Shot Adversarial Robustness in Vision-Language Models
  7. 2024SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
  8. 2026SS-TPT: Stability and Suitability-Guided Test-Time Prompt Tuning for Adversarially Robust Vision-Language Models