mlsanityData to 7 Oct 2026

Obliviate: Neutralizing Task-agnostic Backdoors within the Parameter-efficient Fine-tuning Paradigm

arXiv 2409.14119 ↗First posted 21 Sep 2024cs.CL cs.AI cs.CRcited by 7

Not on the map. Why?

Abstract (arXiv)

Parameter-efficient fine-tuning (PEFT) has become a key training strategy for large language models. However, its reliance on fewer trainable parameters poses security risks, such as task-agnostic backdoors. Despite their severe impact on a wide range of tasks, there is no practical defense solution available that effectively counters task-agnostic backdoors within the context of PEFT. In this study, we introduce Obliviate, a PEFT-integrable backdoor defense. We develop two techniques aimed at amplifying benign neurons within PEFT layers and penalizing the influence of trigger tokens. Our evaluations across three major PEFT architectures show that our method can significantly reduce the attack success rate of the state-of-the-art task-agnostic backdoors (83.6%downarrow). Furthermore, our method exhibits robust defense capabilities against both task-specific backdoors and adaptive attacks. Source code will be obtained at https://github.com/obliviateARR/Obliviate.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 21 Sep 2024.

Builds on

37 of 52 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  11. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  12. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
25 more
  1. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  2. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  3. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  4. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  5. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  6. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 231
  7. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
  8. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  9. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 203
  10. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  11. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 160
  12. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  13. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 103
  14. 2021RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Modelscited by 97
  15. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 82
  16. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  17. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 76
  18. 2019Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networkscited by 66
  19. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classificationcited by 56
  20. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injectioncited by 47
  21. 2021Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attackscited by 44
  22. 2020Backdoor Attacks against Transfer Learning with Pre-trained Deep Learning Modelscited by 43
  23. 2023Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attackscited by 32
  24. 2023Defending against Insertion-based Textual Backdoor Attacks via Attributioncited by 24
  25. 2024DPPA: Pruning Method for Large Language Model to Model Mergingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 3.

7 papers.

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  3. 2024Claim-Guided Textual Backdoor Attack for Practical Applicationscited by 3
  4. 2025SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Modelscited by 1
  5. 2026Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directionscited by 1
  6. 2026Defense Against LLM Backdoors using Critical Neuron Isolation Pruningcited by 0
  7. 2026LoRAScan: Detecting Backdoor Prompts in Low-Rank Adapters for Large Language Models via Down-Projection Activation Spikescited by 0