Exposing Functional Fusion: A New Class of Strategic Backdoor in Dynamic Prompt Architectures
On the map: LLM safety and security → Backdoor and data poisoning attacks
Abstract (arXiv)
Existing ViT backdoor attacks based on backbone-overwriting full-tuning are computationally expensive and inflict performance degradation. This has forced adversaries towards the Visual Parameter-Efficient Fine-Tuning (PEFT) paradigm, dominated by adapter-based (e.g., LoRA) and prompt-based (e.g., VPT) approaches. While adapter security has seen initial study, the risks of the burgeoning prompt-based ecosystem remain critically unexplored. We fill this critical gap, exposing how the evolution of VPT towards dynamic and context-aware architectures can facilitate a far more dangerous and emergent threat. This vulnerability arises even though these dynamic modules unlock superior benign performance. We propose VIPER, an attack framework built on a lightweight, dynamic Visual Prompt Generator (VPG) that demonstrates this vulnerability. Critically, this dynamic architecture enables Functional Fusion: an emergent phenomenon where malicious logic and benign task utility are tightly fused into the same sparse, high-magnitude parameter core. This fusion creates a formidable ``hostage" dilemma, as pruning the attack necessarily destroys the benign performance. Comprehensive evaluations show VIPER effectively addresses the attacker's trilemma: VIPER not only achieves state-of-the-art performance on clean data, but also maintains near-100% ASR even under 90% VPG-module pruning (where LoRA attacks collapse), while adding only an imperceptible 0.06ms (1.16%) of inference latency. VIPER's results, driven by Functional Fusion, expose a new, paradigm-level risk in dynamic prompt architectures.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 24 of its 42 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
24 of 42 references.
Closest ancestors · 2
Same direction, newest first
- 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP26 Nov
- 2022TrojViT: Trojan Insertion in Vision Transformers27 Aug
Other references · 22
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2013Describing Textures in the Wildcited by 2,125
- 2022Visual Prompt Tuningcited by 1,681
- 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
- 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
- 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
- 2019Hidden Trigger Backdoor Attackscited by 371
- 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 353
- 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
10 more
- 2021Incorporating Convolution Designs into Visual Transformerscited by 248
- 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
- 2023Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Modelscited by 68
- 2024Parameter Efficient Fine-tuning of Self-supervised ViTs without Catastrophic Forgettingcited by 17
- 2025DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformerscited by 17
- 2023DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysiscited by 11
- 2024FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformerscited by 9
- 2025PRO-VPT: Distribution-Adaptive Visual Prompt Tuning via Prompt Relocationcited by 9
- 2023Med-Tuning: A New Parameter-Efficient Tuning Framework for Medical Volumetric Segmentationcited by 3
- 2023Context-Aware Prompt Tuning for Vision-Language Model with Dual-Alignmentcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ATAAT: Adaptive Threat-Aware Adversarial Tuning Framework against Backdoor Attacks on Vision-Language-Action Models ↗
- 2026FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Models
- 2023Prompt Backdoors in Visual Prompt Learning
- 2026Inject Once Survive Later: Backdooring Vision-Language-Action Models to Persist Through Downstream Fine-tuning
- 2023LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
- 2024Context is the Key: Backdoor Attacks for In-Context Learning with Vision Transformers
- 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection
- 2026BadBone: Backdoor Attacks Against Backbone Models in Visual Prompt Learning