# A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs

arXiv: https://arxiv.org/abs/2011.03083 · HTML: https://mlsanity.com/paper/2011.03083/

## Key facts

- Cited by 37 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 30 of 37)

- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [A Light Recipe to Train Robust Vision Transformers](https://mlsanity.com/paper/2209.07399/) (2022)
- [HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise](https://mlsanity.com/paper/2110.11417/) (2021)
- [Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack](https://mlsanity.com/paper/2203.05154/) (2022)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference](https://mlsanity.com/paper/2301.09254/) (2023)
- [AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models](https://mlsanity.com/paper/2403.13269/) (2024)
- [Pipeline Parallelism for Inference on Heterogeneous Edge Computing](https://mlsanity.com/paper/2110.14895/) (2021)
- [Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs](https://mlsanity.com/paper/2310.02277/) (2023)
- [Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge](https://mlsanity.com/paper/2208.13092/) (2022)
- [BMPQ: Bit-Gradient Sensitivity Driven Mixed-Precision Quantization of DNNs from Scratch](https://mlsanity.com/paper/2112.13843/) (2021)
- [Neural Network Compression for Noisy Storage Devices](https://mlsanity.com/paper/2102.07725/) (2021)
- [Self-Attentive Pooling for Efficient Deep Learning](https://mlsanity.com/paper/2209.07659/) (2022)
- [Machine Learning Robustness: A Primer](https://mlsanity.com/paper/2404.00897/) (2024)
- [Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression](https://mlsanity.com/paper/2107.12445/) (2021)
- [Robust NAS under adversarial training: benchmark, theory, and beyond](https://mlsanity.com/paper/2403.13134/) (2024)
- [Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction](https://mlsanity.com/paper/2301.03573/) (2023)
- [Holistic Adversarially Robust Pruning](https://mlsanity.com/paper/2412.14714/) (2024)
- [Robustness in Compressed Neural Networks for Object Detection](https://mlsanity.com/paper/2102.05509/) (2021)
- [Deep Neural Networks pruning via the Structured Perspective Regularization](https://mlsanity.com/paper/2206.14056/) (2022)
- [Sparse Mixture Once-for-all Adversarial Training for Efficient In-Situ Trade-Off Between Accuracy and Robustness of DNNs](https://mlsanity.com/paper/2302.03523/) (2022)
- [Making Models Shallow Again: Jointly Learning to Reduce Non-Linearity and Depth for Latency-Efficient Private Inference](https://mlsanity.com/paper/2304.13274/) (2023)
- [Block Selective Reprogramming for On-device Training of Vision Transformers](https://mlsanity.com/paper/2405.10951/) (2024)
- [RBNN: Memory-Efficient Reconfigurable Deep Binary Neural Network with IP Protection for Internet of Things](https://mlsanity.com/paper/2105.03822/) (2021)
- [Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology](https://mlsanity.com/paper/2312.01213/) (2023)
- [Interpretability-Guided Test-Time Adversarial Defense](https://mlsanity.com/paper/2409.15190/) (2024)
- [A Fast and Efficient Conditional Learning for Tunable Trade-Off between Accuracy and Robustness](https://mlsanity.com/paper/2204.00426/) (2022)
- [IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks](https://mlsanity.com/paper/2310.11890/) (2023)
- [ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs](https://mlsanity.com/paper/2410.24214/) (2024)
- [Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models](https://mlsanity.com/paper/2504.04747/) (2025)

## Abstract (arXiv)

This paper presents a dynamic network rewiring (DNR) method to generate pruned deep neural network (DNN) models that are robust against adversarial attacks yet maintain high accuracy on clean images. In particular, the disclosed DNR method is based on a unified constrained optimization formulation using a hybrid loss function that merges ultra-high model compression with robust adversarial training. This training strategy dynamically adjusts inter-layer connectivity based on per-layer normalized momentum computed from the hybrid loss function. In contrast to existing robust pruning frameworks that require multiple training iterations, the proposed learning strategy achieves an overall target pruning ratio with only a single training iteration and can be tuned to support both irregular and structured channel pruning. To evaluate the merits of DNR, experiments were performed with two widely accepted models, namely VGG16 and ResNet-18, on CIFAR-10, CIFAR-100 as well as with VGG16 on Tiny-ImageNet. Compared to the baseline uncompressed models, DNR provides over20x compression on all the datasets with no significant drop in either clean or adversarial classification accuracy. Moreover, our experiments show that DNR consistently finds compressed models with better clean and adversarial image classification performance than what is achievable through state-of-the-art alternatives.