mlsanityData to 14 Oct 2026

BaCP: Backbone Contrastive Pruning for Preserving Representations in Extremely Sparse Neural Networks

arXiv 2610.02524 ↗First posted 1 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Unstructured pruning at extreme sparsity often suffers from representational collapse, causing sharp drops in accuracy. To address this, we study Backbone Contrastive Pruning (BaCP), which regularizes the sparse network's embedding space by aligning it with pretrained, fine-tuned, and historical snapshot models. Building on the contrastive decomposition of the CAP framework (Xu et al., 2022), we provide a rigorous matched-budget characterization of this approach across multiple pruning criteria. Evaluated across 90 settings, BaCP improves accuracy substantially in extreme sparsity regimes where standard pruning fails, and is close to baseline where representations remain intact.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

20 of 26 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2020Supervised Contrastive Learningcited by 2,829
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  7. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  9. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  10. 2020What is the State of Neural Network Pruning?cited by 583
  11. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  12. 2019The State of Sparsity in Deep Neural Networkscited by 476
8 more
  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
  2. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  3. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 84
  4. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 46
  5. 2020Pruning via Iterative Ranking of Sensitivity Statisticscited by 26
  6. 2021From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compressioncited by 19
  7. 2022Studying the impact of magnitude pruning on contrastive learning methodscited by 7
  8. 2024Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruningcited by 1

Built on by

Not cited yet.