# Bridgeout: stochastic bridge regularization for deep neural networks

arXiv: https://arxiv.org/abs/1804.08042 · HTML: https://mlsanity.com/paper/1804.08042/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2018.

## Built on by (most cited first, 4 of 4)

- [Decoupled Weight Decay for Any p Norm](https://mlsanity.com/paper/2404.10824/) (2024)
- [Adaptive Low-Rank Regularization with Damping Sequences to Restrict Lazy Weights in Deep Networks](https://mlsanity.com/paper/2106.09677/) (2021)
- [Global Minimizers of ℓp-Regularized Objectives Yield the Sparsest ReLU Neural Networks](https://mlsanity.com/paper/2505.21791/) (2025)
- [On the Mathematical Impossibility of Safe Universal Approximators](https://mlsanity.com/paper/2507.03031/) (2025)

## Abstract (arXiv)

A major challenge in training deep neural networks is overfitting, i.e. inferior performance on unseen test examples compared to performance on training examples. To reduce overfitting, stochastic regularization methods have shown superior performance compared to deterministic weight penalties on a number of image recognition tasks. Stochastic methods such as Dropout and Shakeout, in expectation, are equivalent to imposing a ridge and elastic-net penalty on the model parameters, respectively. However, the choice of the norm of weight penalty is problem dependent and is not restricted to L1,L2. Therefore, in this paper we propose the Bridgeout stochastic regularization technique and prove that it is equivalent to an Lq penalty on the weights, where the norm q can be learned as a hyperparameter from data. Experimental results show that Bridgeout results in sparse model weights, improved gradients and superior classification performance compared to Dropout and Shakeout on synthetic and real datasets.