# Biologically inspired protection of deep networks from adversarial attacks

arXiv: https://arxiv.org/abs/1703.09202 · HTML: https://mlsanity.com/paper/1703.09202/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2017.

## Built on by (most cited first, 23 of 23)

- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [Adversarial Example Detection for DNN Models: A Review and Experimental Comparison](https://mlsanity.com/paper/2105.00203/) (2021)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Unsupervised Robust Domain Adaptation without Source Data](https://mlsanity.com/paper/2103.14577/) (2021)
- [A Unified Game-Theoretic Interpretation of Adversarial Robustness](https://mlsanity.com/paper/2111.03536/) (2021)
- [Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features](https://mlsanity.com/paper/2111.03740/) (2021)
- [A Person Re-identification Data Augmentation Method with Adversarial Defense Effect](https://mlsanity.com/paper/2101.08783/) (2021)
- [Understanding Robust Learning through the Lens of Representation Similarities](https://mlsanity.com/paper/2206.09868/) (2022)
- [Revisiting Model's Uncertainty and Confidences for Adversarial Example Detection](https://mlsanity.com/paper/2103.05354/) (2021)
- [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022)
- [Computing a human-like reaction time metric from stable recurrent vision models](https://mlsanity.com/paper/2306.11582/) (2023)
- [Jacobian Norm with Selective Input Gradient Regularization for Improved and Interpretable Adversarial Defense](https://mlsanity.com/paper/2207.13036/) (2022)
- [Evasion Adversarial Attacks Remain Impractical Against ML-based Network Intrusion Detection Systems, Especially Dynamic Ones](https://mlsanity.com/paper/2306.05494/) (2023)
- [Tracklet-Switch Adversarial Attack against Pedestrian Multi-Object Tracking Trackers](https://mlsanity.com/paper/2111.08954/) (2021)
- [Enhancing Classifier Conservativeness and Robustness by Polynomiality](https://mlsanity.com/paper/2203.12693/) (2022)
- [Spatial-Frequency Discriminability for Revealing Adversarial Perturbations](https://mlsanity.com/paper/2305.10856/) (2023)
- [A Novel Perturb-ability Score to Mitigate Evasion Adversarial Attacks on Flow-Based ML-NIDS](https://mlsanity.com/paper/2409.07448/) (2024)
- [Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations](https://mlsanity.com/paper/2603.03234/) (2026)
- [And/or trade-off in artificial neurons: impact on adversarial robustness](https://mlsanity.com/paper/2102.07389/) (2021)
- [Stochastic-HMDs: Adversarial Resilient Hardware Malware Detectors through Voltage Over-scaling](https://mlsanity.com/paper/2103.06936/) (2021)
- [A Unified Game-Theoretic Interpretation of Adversarial Robustness](https://mlsanity.com/paper/2103.07364/) (2021)
- [3D-OOCS: Learning Prostate Segmentation with Inductive Bias](https://mlsanity.com/paper/2110.15664/) (2021)
- [Retina gap junctions support the robust perception by warping neural representational geometries along the visual hierarchy](https://mlsanity.com/paper/2604.14200/) (2026)

## Abstract (arXiv)

Inspired by biophysical principles underlying nonlinear dendritic computation in neural circuits, we develop a scheme to train deep neural networks to make them robust to adversarial attacks. Our scheme generates highly nonlinear, saturated neural networks that achieve state of the art performance on gradient based adversarial examples on MNIST, despite never being exposed to adversarially chosen examples during training. Moreover, these networks exhibit unprecedented robustness to targeted, iterative schemes for generating adversarial examples, including second-order methods. We further identify principles governing how these networks achieve their robustness, drawing on methods from information geometry. We find these networks progressively create highly flat and compressed internal representations that are sensitive to very few input dimensions, while still solving the task. Moreover, they employ highly kurtotic weight distributions, also found in the brain, and we demonstrate how such kurtosis can protect even linear classifiers from adversarial attack.