mlsanityData to 14 Oct 2026

Bayesian Compression for Deep Learning

arXiv 1705.08665 ↗First posted 24 May 2017stat.ML cs.LGcited by 116

Not on the map. Why?

Abstract (arXiv)

Compression and computational efficiency in deep learning have become a problem of great significance. In this work, we argue that the most principled and effective way to attack this problem is by adopting a Bayesian point of view, where through sparsity inducing priors we prune large parts of the network. We introduce two novelties in this paper: 1) we use hierarchical priors to prune nodes instead of individual weights, and 2) we use the posterior uncertainties to determine the optimal fixed point precision to encode the weights. Both factors significantly contribute to achieving the state of the art in terms of compression rates, while still staying competitive with methods designed to optimize for speed or energy efficiency.

Key facts. Cited by 116 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 30; 2022: 23; 2023: 20; 2024: 22; 2025: 15; 2026: 6.

116 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  2. 2021COIN: COmpression with Implicit Neural representationscited by 233
  3. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  4. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  5. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Frameworkcited by 70
  6. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 62
  7. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 61
  8. 2021Overfitting for Fun and Profit: Instance-Adaptive Data Compressioncited by 38
  9. 2021Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translationcited by 35
  10. 2023OTOV2: Automatic, Generic, User-Friendlycited by 32
  11. 2022Green Learning: Introduction, Examples and Outlookcited by 30
  12. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
38 more
  1. 2021Selfish Sparse RNN Trainingcited by 26
  2. 2021An Information-Theoretic Justification for Model Pruningcited by 25
  3. 2023Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Trainingcited by 23
  4. 2023ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensemblescited by 20
  5. 2023SpArX: Sparse Argumentative Explanations for Neural Networks [Technical Report]cited by 18
  6. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  7. 2021Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Detailscited by 14
  8. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  9. 2024Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Lengthcited by 14
  10. 2025Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compressioncited by 14
  11. 2022Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibilitycited by 13
  12. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  13. 2021Partition and Code: learning how to compress graphscited by 10
  14. 2021Neural network relief: a pruning algorithm based on neural activitycited by 10
  15. 2024A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learningcited by 10
  16. 2021Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networkscited by 9
  17. 2023Incorporating Unlabelled Data into Bayesian Neural Networkscited by 9
  18. 2023Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inferencecited by 9
  19. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  20. 2024Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimallycited by 9
  21. 2021Neural Network Compression for Noisy Storage Devicescited by 8
  22. 2022CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsificationcited by 8
  23. 2021Deep Model Compression based on the Training Historycited by 7
  24. 2021Structured Dropout Variational Inference for Bayesian Neural Networkscited by 7
  25. 2021Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processescited by 7
  26. 2022Towards Effective and Robust Neural Trojan Defenses via Input Filteringcited by 7
  27. 2022LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsificationcited by 7
  28. 2024Training Bayesian Neural Networks with Sparse Subspace Variational Inferencecited by 7
  29. 2021Single-path Bit Sharing for Automatic Loss-aware Model Compressioncited by 6
  30. 2021Efficacy of Bayesian Neural Networks in Active Learningcited by 6
  31. 2021A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statisticscited by 6
  32. 2022Cyclical Pruning for Sparse Neural Networkscited by 6
  33. 2022Principled Pruning of Bayesian Neural Networks through Variational Free Energy Minimizationcited by 6
  34. 2024Governing equation discovery of a complex system from snapshotscited by 6
  35. 2025Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformerscited by 6
  36. 2022Fast Conditional Network Compression Using Bayesian HyperNetworkscited by 5
  37. 2022Sequential Bayesian Neural Subnetwork Ensemblescited by 5
  38. 2023Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertaintycited by 5

50 most cited of 116 All 116 (JSON)