mlsanityData to 7 Oct 2026

The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Models

arXiv 2003.07233 ↗First posted 13 Mar 2020cs.LG cs.CRcited by 41

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce the TrojAI software framework, an open source set of Python tools capable of generating triggered (poisoned) datasets and associated deep learning (DL) models with trojans at scale. We utilize the developed framework to generate a large set of trojaned MNIST classifiers, as well as demonstrate the capability to produce a trojaned reinforcement-learning model using vector observations. Results on MNIST show that the nature of the trigger, training batch size, and dataset poisoning percentage all affect successful embedding of trojans. We test Neural Cleanse against the trojaned MNIST models and successfully detect anomalies in the trained models approximately 18% of the time. Our experiments and workflow indicate that the TrojAI software framework will enable researchers to easily understand the effects of various configurations of the dataset and training hyperparameters on the generated trojaned deep learning model, and can be used to rapidly and comprehensively test new trojan detection methods.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 5; 2023: 6; 2024: 12; 2025: 8.

41 papers.

Sort:

By area:

  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  2. 2021Unsolved Problems in ML Safetycited by 291
  3. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 124
  4. 2022Training with More Confidence: Mitigating Injected and Natural Backdoors During Trainingcited by 38
  5. 2021Dual-Key Multimodal Backdoors for Visual Question Answeringcited by 35
  6. 2021Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggerscited by 19
  7. 2024BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learningcited by 17
  8. 2022Provable Defense against Backdoor Policies in Reinforcement Learningcited by 16
  9. 2021Trojan Signatures in DNN Weightscited by 15
  10. 2021TOP: Backdoor Detection in Neural Networks via Transferability of Perturbationcited by 13
  11. 2023A Holistic Assessment of the Reliability of Machine Learning Systemscited by 9
  12. 2025Learning to Interpret Weight Differences in Language Modelscited by 9
29 more
  1. 2021TAD: Trigger Approximation based Black-box Trojan Detection for AIcited by 8
  2. 2025BackdoorDM: A Comprehensive Benchmark for Backdoor Learning on Diffusion Modelcited by 7
  3. 2024Rethinking Backdoor Detection Evaluation for Language Modelscited by 6
  4. 2021MISA: Online Defense of Trojaned Models using Misattributionscited by 4
  5. 2023Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineeringcited by 4
  6. 2022PerD: Perturbation Sensitivity-based Neural Trojan Detection Framework on NLP Applicationscited by 3
  7. 2024BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluationcited by 3
  8. 2025Scanning Trojaned Models Using Out-of-Distribution Samplescited by 3
  9. 2021MLDS: A Dataset for Weight-Space Analysis of Neural Networkscited by 2
  10. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  11. 2025Revisiting Training-Inference Trigger Intensity in Backdoor Attackscited by 2
  12. 2021SanitAIs: Unsupervised Data Augmentation to Sanitize Trojaned Neural Networkscited by 1
  13. 2022An Adaptive Black-box Backdoor Detection Method for Deep Neural Networkscited by 1
  14. 2025BackFed: An Efficient & Standardized Benchmark Suite for Backdoor Attacks in Federated Learningcited by 1
  15. 2025DISTIL: Data-Free Inversion of Suspicious Trojan Inputs via Latent Diffusioncited by 1
  16. 2021Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognitioncited by 0
  17. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
  18. 2023GMTR: Graph Matching Transformerscited by 0
  19. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
  20. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  21. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  22. 2024BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learningcited by 0
  23. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  24. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  25. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  26. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0
  27. 2024Solving Trojan Detection Competitions with Linear Weight Classificationcited by 0
  28. 2025DeBUGCN -- Detecting Backdoors in CNNs Using Graph Convolutional Networkscited by 0
  29. 2025Mask6D: Masked Pose Priors For 6D Object Pose Estimationcited by 0