mlsanityData to 7 Oct 2026

Learning Sparse Neural Networks via Sensitivity-Driven Regularization

arXiv 1810.11764 ↗First posted 28 Oct 2018cs.LG cond-mat.dis-nn stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

The ever-increasing number of parameters in deep neural networks poses challenges for memory-limited applications. Regularize-and-prune methods aim at meeting these challenges by sparsifying the network weights. In this context we quantify the output sensitivity to the parameters (i.e. their relevance to the network output) and introduce a regularization term that gradually lowers the absolute value of parameters with low sensitivity. Thus, a very large fraction of the parameters approach zero and are eventually set to zero by simple thresholding. Our method surpasses most of the recent techniques both in terms of sparsity and error rates. In some cases, the method reaches twice the sparsity obtained by other techniques at equal error rates.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 13; 2023: 3; 2024: 2; 2025: 2.

26 papers.

Sort:

  1. 2022Compressing Explicit Voxel Grid Representations: fast NeRFs become also smallcited by 48
  2. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  3. 2021Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generationcited by 14
  4. 2022To update or not to update? Neurons at equilibrium in deep modelscited by 13
  5. 2021SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networkscited by 12
  6. 2024Is Oracle Pruning the True Oracle?cited by 7
  7. 2023Mini but Mighty: Finetuning ViTs with Mini Adapterscited by 6
  8. 2022Pruning-aware Sparse Regularization for Network Pruningcited by 5
  9. 2021HEMP: High-order Entropy Minimization for neural network comPressioncited by 4
  10. 2022Automatic Sparse Connectivity Learning for Neural Networkscited by 4
  11. 2022Amenable Sparse Network Investigatorcited by 3
  12. 2022Regularization-based Pruning of Irrelevant Weights in Deep Neural Architecturescited by 3
14 more
  1. 2022Towards Efficient Capsule Networkscited by 3
  2. 2022Learning sparse auto-encoders for green AI image codingcited by 3
  3. 2022Capsule Networks Do Not Need to Model Everythingcited by 2
  4. 2025Engram Memory Encoding and Retrieval: A Neurocomputational Perspectivecited by 2
  5. 2022Sparse Super-Regular Networkscited by 1
  6. 2022Improve Convolutional Neural Network Pruning by Maximizing Filter Varietycited by 1
  7. 2022VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognitioncited by 1
  8. 2023Near-Linear Time Projection onto the ℓ1,∞ Ball; Application to Sparse Autoencoderscited by 1
  9. 2025Efficient Adaptation of Deep Neural Networks for Semantic Segmentation in Space Applicationscited by 1
  10. 2021MinConvNets: A new class of multiplication-less Neural Networkscited by 0
  11. 2021Training Sparse Neural Network by Constraining Synaptic Weight on Unit Lp Spherecited by 0
  12. 2021Description of Structural Biases and Associated Data in Sensor-Rich Environmentscited by 0
  13. 2023Combining Past, Present and Future: A Self-Supervised Approach for Class Incremental Learningcited by 0
  14. 2024Multi-level projection with exponential parallel speedup; Application to sparse auto-encoders neural networkscited by 0