# LOss-Based SensiTivity rEgulaRization: towards deep sparse neural networks

arXiv: https://arxiv.org/abs/2011.09905 · HTML: https://mlsanity.com/paper/2011.09905/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2020.

## Built on by (most cited first, 19 of 19)

- [Packed-Ensembles for Efficient Uncertainty Estimation](https://mlsanity.com/paper/2210.09184/) (2022)
- [Compressing Explicit Voxel Grid Representations: fast NeRFs become also small](https://mlsanity.com/paper/2210.12782/) (2022)
- [Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?](https://mlsanity.com/paper/2308.06619/) (2023)
- [Adaptive Sharpness-Aware Pruning for Robust Sparse Networks](https://mlsanity.com/paper/2306.14306/) (2023)
- [To update or not to update? Neurons at equilibrium in deep models](https://mlsanity.com/paper/2207.09455/) (2022)
- [Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs](https://mlsanity.com/paper/2107.13490/) (2021)
- [DSD2: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?](https://mlsanity.com/paper/2303.01213/) (2023)
- [HEMP: High-order Entropy Minimization for neural network comPression](https://mlsanity.com/paper/2107.05298/) (2021)
- [Layer Collapse Can be Induced by Unstructured Pruning](https://mlsanity.com/paper/2404.16890/) (2024)
- [Regularization-based Pruning of Irrelevant Weights in Deep Neural Architectures](https://mlsanity.com/paper/2204.04977/) (2022)
- [Towards Efficient Capsule Networks](https://mlsanity.com/paper/2208.09203/) (2022)
- [The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth](https://mlsanity.com/paper/2404.18949/) (2024)
- [Capsule Networks Do Not Need to Model Everything](https://mlsanity.com/paper/2204.01298/) (2022)
- [SkinFormer: Learning Statistical Texture Representation with Transformer for Skin Lesion Segmentation](https://mlsanity.com/paper/2409.08652/) (2024)
- [LaCoOT: Layer Collapse through Optimal Transport](https://mlsanity.com/paper/2406.08933/) (2024)
- [Efficient Adaptation of Deep Neural Networks for Semantic Segmentation in Space Applications](https://mlsanity.com/paper/2504.15991/) (2025)
- [Routing the Lottery: Adaptive Subnetworks for Heterogeneous Data](https://mlsanity.com/paper/2601.22141/) (2026)
- [Learn how to Prune Pixels for Multi-view Neural Image-based Synthesis](https://mlsanity.com/paper/2305.03572/) (2023)
- [From Deep to Shallow: Unconstrained and Efficient Layer Merging Strategy](https://mlsanity.com/paper/2609.04881/) (2026)

## Abstract (arXiv)

LOBSTER (LOss-Based SensiTivity rEgulaRization) is a method for training neural networks having a sparse topology. Let the sensitivity of a network parameter be the variation of the loss function with respect to the variation of the parameter. Parameters with low sensitivity, i.e. having little impact on the loss when perturbed, are shrunk and then pruned to sparsify the network. Our method allows to train a network from scratch, i.e. without preliminary learning or rewinding. Experiments on multiple architectures and datasets show competitive compression ratios with minimal computational overhead.