# Unreasonable Effectiveness of Last Hidden Layer Activations for Adversarial Robustness

arXiv: https://arxiv.org/abs/2202.07342 · HTML: https://mlsanity.com/paper/2202.07342/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 29 of 29)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients](https://mlsanity.com/paper/1711.09404/) (2017)
- [Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors](https://mlsanity.com/paper/1807.07978/) (2018)
- [Privacy Risks of Securing Machine Learning Models against Adversarial Examples](https://mlsanity.com/paper/1905.10291/) (2019)
- [Interpretable Adversarial Perturbation in Input Embedding Space for Text](https://mlsanity.com/paper/1805.02917/) (2018)
- [Adversarial Attacks Against Medical Deep Learning Systems](https://mlsanity.com/paper/1804.05296/) (2018)
- [APE-GAN: Adversarial Perturbation Elimination with GAN](https://mlsanity.com/paper/1707.05474/) (2017)
- [DARTS: Deceiving Autonomous Cars with Toxic Signs](https://mlsanity.com/paper/1802.06430/) (2018)
- [Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward](https://mlsanity.com/paper/1905.12762/) (2019)
- [Fooling a Real Car with Adversarial Traffic Signs](https://mlsanity.com/paper/1907.00374/) (2019)
- [Exploiting epistemic uncertainty of the deep learning models to generate adversarial samples](https://mlsanity.com/paper/2102.04150/) (2021)
- [Closeness and Uncertainty Aware Adversarial Examples Detection in Adversarial Machine Learning](https://mlsanity.com/paper/2012.06390/) (2020)
- [A Generative Model based Adversarial Security of Deep Learning and Linear Classifier Models](https://mlsanity.com/paper/2010.08546/) (2020)
- [Why Blocking Targeted Adversarial Perturbations Impairs the Ability to Learn](https://mlsanity.com/paper/1907.05718/) (2019)

## Built on by (most cited first, 1 of 1)

- [PoTS: Proof-of-Training-Steps for Backdoor Detection in Large Language Models](https://mlsanity.com/paper/2510.15106/) (2025)

## Abstract (arXiv)

In standard Deep Neural Network (DNN) based classifiers, the general convention is to omit the activation function in the last (output) layer and directly apply the softmax function on the logits to get the probability scores of each class. In this type of architectures, the loss value of the classifier against any output class is directly proportional to the difference between the final probability score and the label value of the associated class. Standard White-box adversarial evasion attacks, whether targeted or untargeted, mainly try to exploit the gradient of the model loss function to craft adversarial samples and fool the model. In this study, we show both mathematically and experimentally that using some widely known activation functions in the output layer of the model with high temperature values has the effect of zeroing out the gradients for both targeted and untargeted attack cases, preventing attackers from exploiting the model's loss function to craft adversarial samples. We've experimentally verified the efficacy of our approach on MNIST (Digit), CIFAR10 datasets. Detailed experiments confirmed that our approach substantially improves robustness against gradient-based targeted and untargeted attack threats. And, we showed that the increased non-linearity at the output layer has some additional benefits against some other attack methods like Deepfool attack.