CLASP: Defending Hybrid Large Language Models Against Hidden State Poisoning Attacks
On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language
Abstract (arXiv)
State space models (SSMs) like Mamba have gained significant traction as efficient alternatives to Transformers, achieving linear complexity while maintaining competitive performance. However, Hidden State Poisoning Attacks (HiSPAs), a recently discovered vulnerability that corrupts SSM memory through adversarial strings, pose a critical threat to these architectures and their hybrid variants. Framing the HiSPA mitigation task as a binary classification problem at the token level, we introduce the CLASP model (Classifier Against State Poisoning) to defend against this threat. CLASP exploits distinct patterns in Mamba's block output embeddings (BOEs) and uses an XGBoost classifier to identify malicious tokens with minimal computational overhead. We consider a realistic scenario in which both SSMs and HiSPAs are likely to be used: an LLM screening r'esum'es to identify the best candidates for a role. Evaluated on a corpus of 2,483 r'esum'es totaling 9.5M tokens with controlled injections, CLASP achieves 95.9% token-level F1 score and 99.3% document-level F1 score on malicious tokens detection. Crucially, the model generalizes to unseen attack patterns: under leave-one-out cross-validation, performance remains high (96.9% document-level F1), while under clustered cross-validation with structurally novel triggers, it maintains useful detection capability (91.6% average document-level F1). Operating independently of any downstream model, CLASP processes 1,032 tokens per second with under 4GB VRAM consumption, potentially making it suitable for real-world deployment as a lightweight front-line defense for SSM-based and hybrid architectures. All code and detailed results are available at https://anonymous.4open.science/r/hispikes-91C0.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 29 of its 37 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
29 of 37 references.
Closest ancestors · 7
Same direction, newest first
- 2026Hidden State Poisoning Attacks against Mamba-based Language Models5 Jan
- 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models4 Apr
- 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling11 Jun
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
- 2024Jamba: A Hybrid Transformer-Mamba Language Model28 Mar
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
- 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct
Other references · 22
- 2017Attention Is All You Needcited by 60,361
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
- 2021Carbon Emissions and Large Neural Network Trainingcited by 567
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
- 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
- 2024Zamba: A Compact 7B SSM Hybrid Modelcited by 124
- 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 99
10 more
- 2019Recurrent Neural Networks (RNNs): A gentle Introduction and Overviewcited by 68
- 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?cited by 52
- 2025Defending Against Prompt Injection with DataFiltercited by 35
- 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 33
- 2025Defending against Indirect Prompt Injection by Instruction Detectioncited by 32
- 2024Green AI: Exploring Carbon Footprints, Mitigation Strategies, and Trade Offs in Large Language Model Trainingcited by 15
- 2025PISanitizer: Preventing Prompt Injection to Long-Context LLMs via Prompt Sanitizationcited by 13
- 2025CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitizationcited by 12
- 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMscited by 10
- 2025DRIP: Defending Prompt Injection via Token-wise Representation Editing and Residual Instruction Fusioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025TokenBreak: Bypassing Text Classification Models Through Token Manipulation
- 2025Adversarial Tokenization
- 2025Broken-Token: Filtering Obfuscated Prompts by Counting Characters-Per-Token
- 2024An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks
- 2026SoK: Robustness in Large Language Models against Jailbreak Attacks
- 2026Jailbreaking LLMs Without Gradients or Priors: Effective and Transferable Attacks
- 2025Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense
- 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks