# Model Compression with Adversarial Robustness: A Unified Optimization Framework

arXiv: https://arxiv.org/abs/1902.03538 · HTML: https://mlsanity.com/paper/1902.03538/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2019.

## Built on by (most cited first, 30 of 71)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [AugMax: Adversarial Composition of Random Augmentations for Robust Training](https://mlsanity.com/paper/2110.13771/) (2021)
- [The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training](https://mlsanity.com/paper/2202.02643/) (2022)
- [A Survey on Long-Tailed Visual Recognition](https://mlsanity.com/paper/2205.13775/) (2022)
- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [Sparsity Winning Twice: Better Robust Generalization from More Efficient Training](https://mlsanity.com/paper/2202.09844/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Robust Mixture-of-Expert Training for Convolutional Neural Networks](https://mlsanity.com/paper/2308.10110/) (2023)
- [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://mlsanity.com/paper/2109.02220/) (2021)
- [Robust Predictable Control](https://mlsanity.com/paper/2109.03214/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning](https://mlsanity.com/paper/2401.10862/) (2024)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Pruning for Robust Concept Erasing in Diffusion Models](https://mlsanity.com/paper/2405.16534/) (2024)
- [Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network](https://mlsanity.com/paper/2204.02738/) (2022)
- [Stealthy Backdoors as Compression Artifacts](https://mlsanity.com/paper/2104.15129/) (2021)
- [Robust Lottery Tickets for Pre-trained Language Models](https://mlsanity.com/paper/2211.03013/) (2022)
- [GhostNets on Heterogeneous Devices via Cheap Operations](https://mlsanity.com/paper/2201.03297/) (2022)
- [Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free](https://mlsanity.com/paper/2205.11819/) (2022)
- [Can pruning improve certified robustness of neural networks?](https://mlsanity.com/paper/2206.07311/) (2022)
- [Robust low-rank training via approximate orthonormal constraints](https://mlsanity.com/paper/2306.01485/) (2023)
- [Adversarial Examples Are Not Bugs, They Are Superposition](https://mlsanity.com/paper/2508.17456/) (2025)
- [Pruning Adversarially Robust Neural Networks without Adversarial Examples](https://mlsanity.com/paper/2210.04311/) (2022)
- [AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networks](https://mlsanity.com/paper/2112.02880/) (2021)
- [Fast and Effective Weight Update for Pruned Large Language Models](https://mlsanity.com/paper/2401.02938/) (2024)
- [Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness](https://mlsanity.com/paper/2206.07839/) (2022)
- [A Secure and Efficient Federated Learning Framework for NLP](https://mlsanity.com/paper/2201.11934/) (2022)
- [Learning Transferable 3D Adversarial Cloaks for Deep Trained Detectors](https://mlsanity.com/paper/2104.11101/) (2021)
- [Integer-arithmetic-only Certified Robustness for Quantized Neural Networks](https://mlsanity.com/paper/2108.09413/) (2021)
- [A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models](https://mlsanity.com/paper/2210.05211/) (2022)

## Abstract (arXiv)

Deep model compression has been extensively studied, and state-of-the-art methods can now achieve high compression ratios with minimal accuracy loss. This paper studies model compression through a different lens: could we compress models without hurting their robustness to adversarial attacks, in addition to maintaining accuracy? Previous literature suggested that the goals of robustness and compactness might sometimes contradict. We propose a novel Adversarially Trained Model Compression (ATMC) framework. ATMC constructs a unified constrained optimization formulation, where existing compression means (pruning, factorization, quantization) are all integrated into the constraints. An efficient algorithm is then developed. An extensive group of experiments are presented, demonstrating that ATMC obtains remarkably more favorable trade-off among model size, accuracy and robustness, over currently available alternatives in various settings. The codes are publicly available at: https://github.com/shupenggui/ATMC.