# Interactive Simulations of Backdoors in Neural Networks

arXiv: https://arxiv.org/abs/2405.13217 · HTML: https://mlsanity.com/paper/2405.13217/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 17 of 17)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [BitNet: Scaling 1-bit Transformers for Large Language Models](https://mlsanity.com/paper/2310.11453/) (2023)
- [Blind Backdoors in Deep Learning Models](https://mlsanity.com/paper/2005.03823/) (2020)
- [TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems](https://mlsanity.com/paper/1908.01763/) (2019)
- [Planting Undetectable Backdoors in Machine Learning Models](https://mlsanity.com/paper/2204.06974/) (2022)
- [A Game-Based Approximate Verification of Deep Neural Networks with Provable Guarantees](https://mlsanity.com/paper/1807.03571/) (2018)
- [Quantization Backdoors to Deep Learning Commercial Frameworks](https://mlsanity.com/paper/2108.09187/) (2021)
- [ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms](https://mlsanity.com/paper/2302.11408/) (2023)
- [Direct-Manipulation Visualization of Deep Networks](https://mlsanity.com/paper/1708.03788/) (2017)
- [A LLM Assisted Exploitation of AI-Guardian](https://mlsanity.com/paper/2307.15008/) (2023)
- [Architectural Neural Backdoors from First Principles](https://mlsanity.com/paper/2402.06957/) (2024)
- [Provably Robust Adversarial Examples](https://mlsanity.com/paper/2007.12133/) (2020)
- [A Survey of Data Security: Practices from Cybersecurity and Challenges of Machine Learning](https://mlsanity.com/paper/2310.04513/) (2023)
- [Game of Trojans: Adaptive Adversaries Against Output-based Trojaned-Model Detectors](https://mlsanity.com/paper/2402.08695/) (2024)

## Abstract (arXiv)

This work addresses the problem of planting and defending cryptographic-based backdoors in artificial intelligence (AI) models. The motivation comes from our lack of understanding and the implications of using cryptographic techniques for planting undetectable backdoors under theoretical assumptions in the large AI model systems deployed in practice. Our approach is based on designing a web-based simulation playground that enables planting, activating, and defending cryptographic backdoors in neural networks (NN). Simulations of planting and activating backdoors are enabled for two scenarios: in the extension of NN model architecture to support digital signature verification and in the modified architectural block for non-linear operators. Simulations of backdoor defense against backdoors are available based on proximity analysis and provide a playground for a game of planting and defending against backdoors. The simulations are available at https://pages.nist.gov/nn-calculator