Prompting the Unseen: Detecting Hidden Backdoors in Black-Box Models
On the map: LLM safety and security → Backdoor and data poisoning attacks
Abstract (arXiv)
Visual prompting (VP) is a new technique that adapts well-trained frozen models for source domain tasks to target domain tasks. This study examines VP's benefits for black-box model-level backdoor detection. The visual prompt in VP maps class subspaces between source and target domains. We identify a misalignment, termed class subspace inconsistency, between clean and poisoned datasets. Based on this, we introduce textscBProm, a black-box model-level detection method to identify backdoors in suspicious models, if any. textscBProm leverages the low classification accuracy of prompted models when backdoors are present. Extensive experiments confirm textscBProm's effectiveness.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 64 of its 84 references that are in the map. First posted on arXiv 14 Nov 2024.
Builds on
64 of 84 references.
Closest ancestors · 3
Same direction, newest first
- 2023Run-Off Election: Improved Provable Defense against Data Poisoning Attacks5 Feb
- 2021Trigger Hunting with a Topological Prior for Trojan Detection15 Oct
- 2021Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective7 Apr
Other references · 61
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
- 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
- 2018Spectral Signatures in Backdoor Attackscited by 457
- 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
- 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
- 2020Backdoor Learning: A Surveycited by 412
- 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
38 more
- 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
- 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
- 2020Input-Aware Dynamic Backdoor Attackcited by 304
- 2019Label-Consistent Backdoor Attackscited by 285
- 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
- 2022Visual Prompting via Image Inpaintingcited by 248
- 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
- 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
- 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
- 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
- 2022Backdoor Defense via Decoupling the Training Processcited by 149
- 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
- 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
- 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 123
- 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
- 2018Adversarial Reprogramming of Neural Networkscited by 115
- 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 93
- 2023SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistencycited by 93
- 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
- 2022Understanding and Improving Visual Prompting: A Label-Mapping Perspectivecited by 86
- 2020Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resourcescited by 83
- 2021Backdoor Attack in the Physical Worldcited by 82
- 2022BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learningcited by 82
- 2020DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentationcited by 75
- 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
- 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 66
- 2022Model Reprogramming: Resource-Efficient Cross-Domain Machine Learningcited by 64
- 2022The "Beatrix'' Resurrections: Robust Backdoor Detection via Gram Matricescited by 62
- 2023Reconstructive Neuron Pruning for Backdoor Defensecited by 56
- 2022MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statisticcited by 53
- 2021AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysiscited by 52
- 2021Poison Ink: Robust and Invisible Backdoor Attackcited by 50
- 2023Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistencycited by 45
- 2023Black-box Backdoor Defense via Zero-shot Image Purificationcited by 44
- 2022Towards A Proactive ML Approach for Detecting Backdoor Poison Samplescited by 40
- 2023Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examplescited by 40
- 2024IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistencycited by 38
- 2023Neural Polarizer: A Lightweight and Effective Backdoor Defense via Purifying Poisoned Featurescited by 35
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.