mlsanityData to 7 Oct 2026

Rethinking the Reverse-engineering of Trojan Triggers

arXiv 2210.15127 ↗First posted 27 Oct 2022cs.CR cs.AI cs.CVcited by 37

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Deep Neural Networks are vulnerable to Trojan (or backdoor) attacks. Reverse-engineering methods can reconstruct the trigger and thus identify affected models. Existing reverse-engineering methods only consider input space constraints, e.g., trigger size in the input space. Expressly, they assume the triggers are static patterns in the input space and fail to detect models with feature space triggers such as image style transformations. We observe that both input-space and feature-space Trojans are associated with feature space hyperplanes. Based on this observation, we design a novel reverse-engineering method that exploits the feature space constraint to reverse-engineer Trojan triggers. Results on four datasets and seven different attacks demonstrate that our solution effectively defends both input-space and feature-space Trojans. It outperforms state-of-the-art reverse-engineering methods and other types of defenses in both Trojaned model detection and mitigation tasks. On average, the detection accuracy of our method is 93%. For Trojan mitigation, our method can reduce the ASR (attack success rate) to only 0.26% with the BA (benign accuracy) remaining nearly unchanged. Our code can be found at https://github.com/RU-System-Software-and-Security/FeatureRE.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 46 of its 89 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

46 of 89 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense11 Feb
  2. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer14 Oct
  3. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning1 Aug
  4. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger26 May
  5. 2021Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models29 Mar

Other references · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2016Wide Residual Networkscited by 3,057
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  11. 2016Universal adversarial perturbationscited by 774
  12. 2018How To Backdoor Federated Learningcited by 745
29 more
  1. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  2. 2018Spectral Signatures in Backdoor Attackscited by 457
  3. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  4. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  5. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  6. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  7. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  8. 2019Label-Consistent Backdoor Attackscited by 285
  9. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  10. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  11. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  12. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  13. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  14. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 160
  15. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  16. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  17. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  18. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  19. 2020Backdoor Attacks to Graph Neural Networkscited by 126
  20. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  21. 2020Graph Backdoorcited by 112
  22. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
  23. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 93
  24. 2020Deep Feature Space Trojan Attack of Neural Networks by Controlled Detoxificationcited by 93
  25. 2022BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learningcited by 82
  26. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 66
  27. 2021PipAttack: Poisoning Federated Recommender Systems forManipulating Item Promotioncited by 37
  28. 2020Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencodercited by 27
  29. 2020FaceHack: Triggering backdoored facial recognition systems using facial characteristicscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 17; 2025: 3; 2026: 9.

37 papers.

Sort:

By area:

  1. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 50
  2. 2023UNICORN: A Unified Backdoor Trigger Inversion Frameworkcited by 44
  3. 2024IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistencycited by 38
  4. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 33
  5. 2023Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shiftcited by 31
  6. 2022FedTracker: Furnishing Ownership Verification and Traceability for Federated Learning Modelcited by 25
  7. 2024Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attackscited by 14
  8. 2024LOTUS: Evasive and Resilient Backdoor Attacks through Sub-Partitioningcited by 13
  9. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
  10. 2023BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detectioncited by 11
  11. 2023What can Discriminator do? Towards Box-free Ownership Verification of Generative Adversarial Networkcited by 10
  12. 2024BAN: Detecting Backdoors Activated by Adversarial Neuron Noisecited by 7
25 more
  1. 2023Certifiably Robust Graph Contrastive Learningcited by 5
  2. 2024Invisible Backdoor Attack against Self-supervised Learningcited by 5
  3. 2024Data-centric NLP Backdoor Defense from the Lens of Memorizationcited by 5
  4. 2024Data Free Backdoor Attackscited by 5
  5. 2023Alteration-free and Model-agnostic Origin Attribution of Generated Imagescited by 4
  6. 2024Defending Deep Regression Models against Backdoor Attackscited by 3
  7. 2024BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluationcited by 3
  8. 2025Improving the Sensitivity of Backdoor Detectors via Class Subspace Orthogonalizationcited by 3
  9. 2025Revisiting Training-Inference Trigger Intensity in Backdoor Attackscited by 2
  10. 2026AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agentscited by 2
  11. 2024CEPA: Consensus Embedded Perturbation for Agnostic Detection and Inversion of Backdoorscited by 1
  12. 2024Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Taskscited by 1
  13. 2024Magnitude-based Neuron Pruning for Backdoor Defenscited by 1
  14. 2024NoiseAttack: An Evasive Sample-Specific Multi-Targeted Backdoor Attack Through White Gaussian Noisecited by 1
  15. 2024Unlearn to Relearn Backdoors: Deferred Backdoor Functionality Attacks on Deep Learning Modelscited by 1
  16. 2024MADE: Graph Backdoor Defense with Masked Unlearningcited by 1
  17. 2025DISTIL: Data-Free Inversion of Suspicious Trojan Inputs via Latent Diffusioncited by 1
  18. 2026BadLLM-TG: A Backdoor Defender powered by LLM Trigger Generatorcited by 0
  19. 2026STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profilingcited by 0
  20. 2026CLIP-Inspector: Model-Level Backdoor Detection for Prompt-Tuned CLIP via OOD Trigger Inversioncited by 0
  21. 2026BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learningcited by 0
  22. 2026Lightweight and Fast Backdoor Model Detectioncited by 0
  23. 2026Fast and Lightweight Backdoor Detection via Head Random Probingcited by 0
  24. 2026Detecting Trojaned DNNs via Spectral Regression Analysiscited by 0
  25. 2026CSO-LLM: Class Subspace Orthogonalization for Post-Training Backdoor Detection and Trigger Inversion in LLMscited by 0