Backdoor Defense through Self-Supervised and Generative Learning
On the map: LLM safety and security → Backdoor and data poisoning attacks
Abstract (arXiv)
Backdoor attacks change a small portion of training data by introducing hand-crafted triggers and rewiring the corresponding labels towards a desired target class. Training on such data injects a backdoor which causes malicious inference in selected test samples. Most defenses mitigate such attacks through various modifications of the discriminative learning procedure. In contrast, this paper explores an approach based on generative modelling of per-class distributions in a self-supervised representation space. Interestingly, these representations get either preserved or heavily disturbed under recent backdoor attacks. In both cases, we find that per-class generative models allow to detect poisoned data and cleanse the dataset. Experiments show that training on cleansed dataset greatly reduces the attack success rate and retains the accuracy on benign inputs.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 52 of its 68 references that are in the map. First posted on arXiv 2 Sep 2024.
Builds on
52 of 68 references.
Closest ancestors · 1
Same direction, newest first
- 2023Rethinking Backdoor Attacks19 Jul
Other references · 51
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2016Density estimation using Real NVPcited by 2,029
- 2020Shortcut Learning in Deep Neural Networkscited by 1,928
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
38 more
- 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
- 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
- 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
- 2018Spectral Signatures in Backdoor Attackscited by 457
- 2018Do Deep Generative Models Know What They Don't Know?cited by 443
- 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
- 2020Backdoor Learning: A Surveycited by 412
- 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
- 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
- 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
- 2020Input-Aware Dynamic Backdoor Attackcited by 304
- 2019Label-Consistent Backdoor Attackscited by 285
- 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
- 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
- 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
- 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
- 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
- 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
- 2021Poisoning and Backdooring Contrastive Learningcited by 149
- 2022Backdoor Defense via Decoupling the Training Processcited by 149
- 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
- 2023SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistencycited by 93
- 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 82
- 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
- 2016Achieving Human Parity in Conversational Speech Recognitioncited by 63
- 2023Reconstructive Neuron Pruning for Backdoor Defensecited by 56
- 2021Certified Patch Robustness via Smoothed Vision Transformerscited by 44
- 2023Backdoor Defense via Adaptively Splitting Poisoned Datasetcited by 43
- 2023Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examplescited by 40
- 2023Backdoor Defense via Deconfounded Representation Learningcited by 38
- 2023Neural Polarizer: A Lightweight and Effective Backdoor Defense via Purifying Poisoned Featurescited by 35
- 2023Towards Stable Backdoor Purification through Feature Shift Tuningcited by 30
- 2023Distilling Cognitive Backdoor Patterns within an Imagecited by 20
- 2022Backdoor Cleansing with Unlabeled Datacited by 17
- 2023CBD: A Certified Backdoor Detector Based on Local Dominant Probabilitycited by 13
- 2023Beating Backdoor Attack at Its Own Gamecited by 11
- 2022Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attackscited by 10
- 2021Incompatibility Clustering as a Defense Against Backdoor Poisoning Attackscited by 5
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.