Data to 7 Oct 2026

Posterior Concentration for Sparse Deep Learning

arXiv 1803.09138 ↗First posted 24 Mar 2018stat.ML cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Spike-and-Slab Deep Learning (SS-DL) is a fully Bayesian alternative to Dropout for improving generalizability of deep ReLU networks. This new type of regularization enables provable recovery of smooth input-output maps with unknown levels of smoothness. Indeed, we show that the posterior distribution concentrates at the near minimax rate for α-H"older smooth maps, performing as well as if we knew the smoothness level α ahead of time. Our result sheds light on architecture design for deep neural networks, namely the choice of depth, width and sparsity level. These network attributes typically depend on unknown smoothness in order to be optimal. We obviate this constraint with the fully Bayes construction. As an aside, we show that SS-DL does not overfit in the sense that the posterior concentrates on smaller networks with fewer (up to the optimal number of) nodes and links. Our results provide new theoretical justifications for deep ReLU networks from a Bayesian point of view.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 10; 2023: 14; 2024: 9; 2025: 7; 2026: 3.

50 papers.

Sort:

By area:

2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
2022Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constantscited by 24
2022VICE: Variational Interpretable Concept Embeddingscited by 15
2021Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Detailscited by 14
2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
2023Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inferencecited by 9
2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 9
2021Sparse Deep Learning: A New Framework Immune to Local Traps and Miscalibrationcited by 7
2022On the inability of Gaussian process regression to optimally learn compositional functionscited by 7

38 more

2022Asymptotic Properties for Bayesian Neural Network in Besov Spacecited by 7
2024Misclassification bounds for PAC-Bayesian sparse deep learningcited by 7
2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clusteringcited by 5
2023Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertaintycited by 5
2024Deep Horseshoe Gaussian Processescited by 5
2021Merging Two Cultures: Deep and Statistical Learningcited by 4
2022Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guaranteescited by 4
2023FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropoutcited by 4
2023Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networkscited by 4
2024Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundscited by 4
2025Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wildcited by 4
2022Bayesian Neural Networks for Macroeconomic Analysiscited by 3
2023Sparse Deep Learning for Time Series Data: Theory and Applicationscited by 3
2024Generative Bayesian Computation for Maximum Expected Utilitycited by 3
2021Approximate Bayesian Computation via Classificationcited by 2
2022A PAC-Bayes oracle inequality for sparse neural networkscited by 2
2023Sparsifying Bayesian neural networks with latent binary variables and normalizing flowscited by 2
2023Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representationscited by 2
2024Generative Modeling: A Reviewcited by 2
2025Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionalitycited by 2
2022Masked Bayesian Neural Networks : Computation and Optimalitycited by 1
2022Deep Partial Least Squares for Empirical Asset Pricingcited by 1
2023Projective Integral Updates for High-Dimensional Variational Inferencecited by 1
2023Personalized Federated Learning via Amortized Bayesian Meta-Learningcited by 1
2024Stochastic Gradient Piecewise Deterministic Monte Carlo Samplerscited by 1
2025Bayesian Coreset Optimization for Personalized Federated Learningcited by 1
2025Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communicationscited by 1
2021Feature Flow Regularization: Improving Structured Sparsity in Deep Neural Networkscited by 0
2023Spectral Gap Regularization of Neural Networkscited by 0
2023The surrogate Gibbs-posterior of a corrected stochastic MALA: Towards uncertainty quantification for neural networkscited by 0
2023A statistical perspective on algorithm unrolling models for inverse problemscited by 0
2024Variational Bayesian Bow tie Neural Networks with Shrinkagecited by 0
2025Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networkscited by 0
2025Generative Learner for Distributional Causal Effectscited by 0
2025Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Priorcited by 0
2026Posterior Concentration of Bayesian Physics-Informed Neural Networks for Elliptic PDEscited by 0
2026Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inferencecited by 0
2026Improved Analysis for Hessian-free High-resolution Monte Carlo Samplingcited by 0