# JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks

arXiv: https://arxiv.org/abs/1904.03750 · HTML: https://mlsanity.com/paper/1904.03750/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2019.

## Built on by (most cited first, 14 of 14)

- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders](https://mlsanity.com/paper/2407.14435/) (2024)
- [Improving Dictionary Learning with Gated Sparse Autoencoders](https://mlsanity.com/paper/2404.16014/) (2024)
- [Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders](https://mlsanity.com/paper/2410.20526/) (2024)
- [A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models](https://mlsanity.com/paper/2407.02646/) (2024)
- [Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models](https://mlsanity.com/paper/2411.14257/) (2024)
- [Route Sparse Autoencoder to Interpret Large Language Models](https://mlsanity.com/paper/2503.08200/) (2025)
- [Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words](https://mlsanity.com/paper/2501.06254/) (2025)
- [Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations](https://mlsanity.com/paper/2502.18147/) (2025)
- [Adaptive Sparse Allocation with Mutual Choice & Feature Choice Sparse Autoencoders](https://mlsanity.com/paper/2411.02124/) (2024)
- [Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoders](https://mlsanity.com/paper/2508.09363/) (2025)
- [Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines](https://mlsanity.com/paper/2605.31183/) (2026)
- [Sparsification and Reconstruction from the Perspective of Representation Geometry](https://mlsanity.com/paper/2505.22506/) (2025)
- [Towards Atoms of Large Language Models](https://mlsanity.com/paper/2509.20784/) (2025)

## Abstract (arXiv)

It has been demonstrated that very simple attacks can fool highly-sophisticated neural network architectures. In particular, so-called adversarial examples, constructed from perturbations of input data that are small or imperceptible to humans but lead to different predictions, may lead to an enormous risk in certain critical applications. In light of this, there has been a great deal of work on developing adversarial training strategies to improve model robustness. These training strategies are very expensive, in both human and computational time. To complement these approaches, we propose a very simple and inexpensive strategy which can be used to ``retrofit'' a previously-trained network to improve its resilience to adversarial attacks. More concretely, we propose a new activation function---the JumpReLU---which, when used in place of a ReLU in an already-trained model, leads to a trade-off between predictive accuracy and robustness. This trade-off is controlled by the jump size, a hyper-parameter which can be tuned during the validation stage. Our empirical results demonstrate that this increases model robustness, protecting against adversarial attacks with substantially increased levels of perturbations. This is accomplished simply by retrofitting existing networks with our JumpReLU activation function, without the need for retraining the model. Additionally, we demonstrate that adversarially trained (robust) models can greatly benefit from retrofitting.