Convergence, design and training of continuous-time dropout as a random batch method
Not on the map. Why?
Abstract (arXiv)
We study continuous-time dropout in controlled differential equations. We introduce a random-batch approximation of additive vector fields. On each time interval of length h, a random subset of components is activated and rescaled by its inclusion probabilities, yielding an unbiased approximation of the full field. We prove a uniform-in-time mean-square trajectory error of order mathcal O(h). At the distribution level, we derive Wasserstein and pointwise density estimates, together with global L1 bounds under moment assumptions. For supervised training, we prove uniform mathcal O(√ h) root-mean-square fluctuations of the randomized objective and an mathcal O(h) weak error for its expectation, leading to consistency of optimal values and near-minimizers and, under quadratic growth, convergence in control space. We further characterize how the sampling law affects the error constant through inclusion probabilities and field variance, compare fixed-size sampling, fixed partitions, and Bernoulli dropout, and derive a work--accuracy model for selecting the switching scale. Numerical experiments with neural ODEs illustrate trajectory and transport errors, sampling effects, and objective consistency at a fixed control.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 72 references that are in the map. First posted on arXiv 15 Oct 2025.
Builds on
44 of 72 references.
References · 44
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
- 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
32 more
- 2016Pruning Filters for Efficient ConvNetscited by 1,129
- 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
- 2015Variational Dropout and the Local Reparameterization Trickcited by 489
- 2017Stable Architectures for Deep Neural Networkscited by 340
- 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 325
- 2016Dynamic Network Surgery for Efficient DNNscited by 233
- 2017Concrete Dropoutcited by 227
- 2014Efficient Object Localization Using Convolutional Networkscited by 226
- 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
- 2019Neural Jump Stochastic Differential Equationscited by 147
- 2020Accelerating Training of Transformer-Based Language Models with Progressive Layer Droppingcited by 93
- 2017Maximum Principle Based Algorithms for Deep Learningcited by 77
- 2018A Mean-Field Optimal Control Formulation of Deep Learningcited by 67
- 2016Error bounds, quadratic growth, and linear convergence of proximal methodscited by 59
- 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
- 2017Structured Bayesian Pruning via Log-Normal Multiplicative Noisecited by 39
- 2019Deep neural networks, generic universal interpolation, and controlled ODEscited by 28
- 2022Implicit regularization of dropoutcited by 24
- 2020Notes on the Behavior of MC Dropoutcited by 21
- 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
- 2020On Convergence and Generalization of Dropout Trainingcited by 19
- 2020Large-time asymptotics in deep learningcited by 14
- 2021Sparse Flows: Pruning Continuous-depth Modelscited by 13
- 2018Stochastic Training of Residual Networks: a Differential Equation Viewpointcited by 12
- 2022Turnpike in optimal control of PDEs, ResNets, and beyondcited by 10
- 2024Constructive approximate transport maps with normalizing flowscited by 4
- 2022Stability and Convergence of a Randomized Model Predictive Control Strategycited by 3
- 2025Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regimecited by 3
- 2021A framework for randomized time-splitting in linear-quadratic optimal controlcited by 2
- 2023Control in finite and infinite dimensioncited by 1
- 2024Mini-batch descent in semiflowscited by 1
- 2025Random Batch Methods for Discretized PDEs on Graphscited by 1
Built on by
Not cited yet.