mlsanityData to 7 Oct 2026

Challenges and Opportunities in Approximate Bayesian Deep Learning for Intelligent IoT Systems

arXiv 2112.01675 ↗First posted 3 Dec 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Approximate Bayesian deep learning methods hold significant promise for addressing several issues that occur when deploying deep learning components in intelligent systems, including mitigating the occurrence of over-confident errors and providing enhanced robustness to out of distribution examples. However, the computational requirements of existing approximate Bayesian inference methods can make them ill-suited for deployment in intelligent IoT systems that include lower-powered edge devices. In this paper, we present a range of approximate Bayesian inference methods for supervised deep learning and highlight the challenges and opportunities when applying these methods on current edge hardware. We highlight several potential solutions to decreasing model storage requirements and improving computational scalability, including model pruning and distillation methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 81 references that are in the map. First posted on arXiv 3 Dec 2021.

Builds on

43 of 81 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2016Pruning Filters for Efficient ConvNetscited by 1,129
31 more
  1. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  2. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  3. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  4. 2020What is the State of Neural Network Pruning?cited by 578
  5. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  6. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  7. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  8. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  9. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  10. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  11. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  12. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  13. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  14. 2019Ensemble Distribution Distillationcited by 158
  15. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  16. 2017Bayesian Compression for Deep Learningcited by 116
  17. 2016R'enyi Divergence Variational Inferencecited by 104
  18. 2019Subspace Inference for Bayesian Deep Learningcited by 94
  19. 2016Learning the Number of Neurons in Deep Networkscited by 69
  20. 2020The Case for Bayesian Deep Learningcited by 68
  21. 2017Bayesian Hypernetworkscited by 64
  22. 2017Model Selection in Bayesian Neural Networks via Horseshoe Priorscited by 52
  23. 2015Bayesian Dark Knowledgecited by 45
  24. 2018Adversarial Distillation of Bayesian Neural Network Posteriorscited by 27
  25. 2016Training Skinny Deep Neural Networks with Iterative Hard Thresholding Methodscited by 15
  26. 2020Generalized Bayesian Posterior Expectation Distillation for Deep Neural Networkscited by 14
  27. 2020URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networkscited by 9
  28. 2021Post-hoc loss-calibration for Bayesian neural networkscited by 9
  29. 2016DSD: Dense-Sparse-Dense Training for Deep Neural Networkscited by 7
  30. 2017Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learningcited by 5
  31. 2020Assessing the Adversarial Robustness of Monte Carlo and Distillation Methods for Deep Bayesian Neural Network Classificationcited by 3

Built on by

Not cited yet.