# MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification

arXiv: https://arxiv.org/abs/2406.05927 · HTML: https://mlsanity.com/paper/2406.05927/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 9 Jun 2024.

## Builds on (most cited first, 30 of 37)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://mlsanity.com/paper/2010.03593/) (2020)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://mlsanity.com/paper/1901.08846/) (2019)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [On the Convergence and Robustness of Adversarial Training](https://mlsanity.com/paper/2112.08304/) (2021)
- [Curriculum Adversarial Training](https://mlsanity.com/paper/1805.04807/) (2018)
- [Smooth Adversarial Training](https://mlsanity.com/paper/2006.14536/) (2020)
- [A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking](https://mlsanity.com/paper/2302.14301/) (2023)
- [CAT: Customized Adversarial Training for Improved Robustness](https://mlsanity.com/paper/2002.06789/) (2020)
- [Efficient Defenses Against Adversarial Attacks](https://mlsanity.com/paper/1707.06728/) (2017)
- [Robust Principles: Architectural Design Principles for Adversarially Robust CNNs](https://mlsanity.com/paper/2308.16258/) (2023)
- [Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies](https://mlsanity.com/paper/2404.09349/) (2024)
- [Low Curvature Activations Reduce Overfitting in Adversarial Training](https://mlsanity.com/paper/2102.07861/) (2021)
- [Parameterizing Activation Functions for Adversarial Robustness](https://mlsanity.com/paper/2110.05626/) (2021)
- [Reproducing Activation Function for Deep Learning](https://mlsanity.com/paper/2101.04844/) (2021)

## Built on by (most cited first, 21 of 21)

- [One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIP](https://mlsanity.com/paper/2505.19840/) (2025)
- [Instant Adversarial Purification with Adversarial Consistency Distillation](https://mlsanity.com/paper/2408.17064/) (2024)
- [CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification](https://mlsanity.com/paper/2502.18176/) (2025)
- [Monitoring Robustness and Individual Fairness](https://mlsanity.com/paper/2506.00496/) (2025)
- [DBLP: Noise Bridge Consistency Distillation For Efficient And Reliable Adversarial Purification](https://mlsanity.com/paper/2508.00552/) (2025)
- [MANI-Pure: Magnitude-Adaptive Noise Injection for Adversarial Purification](https://mlsanity.com/paper/2509.25082/) (2025)
- [A Versatile Framework for Designing Group-Sparse Adversarial Attacks](https://mlsanity.com/paper/2510.16637/) (2025)
- [Diffusion Models are Robust Pretrainers](https://mlsanity.com/paper/2511.02793/) (2025)
- [Secure Diagnostics: Adversarial Robustness Meets Clinical Interpretability](https://mlsanity.com/paper/2504.05483/) (2025)
- [Keep It Real: Challenges in Attacking Compression-Based Adversarial Purification](https://mlsanity.com/paper/2508.05489/) (2025)
- [Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networks](https://mlsanity.com/paper/2510.10000/) (2025)
- [Dynamic Parameter Optimization for Highly Transferable Transformation-Based Attacks](https://mlsanity.com/paper/2511.11993/) (2025)
- [Enhancing Adversarial Robustness with Signed Distance Fields for Harmonizing Geometric Invariance and Texture](https://mlsanity.com/paper/2602.05175/) (2026)
- [Fast Preemptive Robustification: High-Frequency Response Anti-Aligns Shared Vulnerability](https://mlsanity.com/paper/2603.25244/) (2026)
- [Compression as an Adversarial Amplifier Through Decision Space Reduction](https://mlsanity.com/paper/2604.06954/) (2026)
- [Toward Understanding Adversarial Distillation: Why Robust Teachers Fail](https://mlsanity.com/paper/2605.21999/) (2026)
- [Sensitivity as a Double-Edged Sword: A Trade-off Between Discriminability and Adversarial Robustness](https://mlsanity.com/paper/2606.01746/) (2026)
- [A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs](https://mlsanity.com/paper/2606.02267/) (2026)
- [VanillaBench: The Hidden Accuracy Cost of Adversarial Robustness](https://mlsanity.com/paper/2607.12545/) (2026)
- [Neither Adversarial Training Nor Purification: Emergent Adversarial Robustness from Oscillatory Predictive Learning](https://mlsanity.com/paper/2609.08683/) (2026)
- [Frame the adversary: a structure-aware attack methodology](https://mlsanity.com/paper/2609.31128/) (2026)

## Abstract (arXiv)

We present a simple yet effective method to improve the robustness of both Convolutional and attention-based Neural Networks against adversarial examples by post-processing an adversarially trained model. Our technique, MeanSparse, cascades the activation functions of a trained model with novel operators that sparsify mean-centered feature vectors. This is equivalent to reducing feature variations around the mean, and we show that such reduced variations merely affect the model's utility, yet they strongly attenuate the adversarial perturbations and decrease the attacker's success rate. Our experiments show that, when applied to the top models in the RobustBench leaderboard, MeanSparse achieves a new robustness record of 75.28% (from 73.71%), 44.78% (from 42.67%) and 62.12% (from 59.56%) on CIFAR-10, CIFAR-100 and ImageNet, respectively, in terms of AutoAttack accuracy. Code is available at https://github.com/SPIN-UMass/MeanSparse