mlsanityData to 7 Oct 2026

An Optimal Time Variable Learning Framework for Deep Neural Networks

arXiv 2204.08528 ↗First posted 18 Apr 2022math.OC cs.LG cs.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

Feature propagation in Deep Neural Networks (DNNs) can be associated to nonlinear discrete dynamical systems. The novelty, in this paper, lies in letting the discretization parameter (time step-size) vary from layer to layer, which needs to be learned, in an optimization framework. The proposed framework can be applied to any of the existing networks such as ResNet, DenseNet or Fractional-DNN. This framework is shown to help overcome the vanishing and exploding gradient issues. Stability of some of the existing continuous DNNs such as Fractional-DNN is also studied. The proposed approach is applied to an ill-posed 3D-Maxwell's equation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 63 references that are in the map. First posted on arXiv 18 Apr 2022.

Builds on

28 of 63 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  5. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  6. 2016Deep Convolutional Neural Network for Inverse Problems in Imagingcited by 431
  7. 2019Augmented Neural ODEscited by 396
  8. 2017Learning a Variational Network for Reconstruction of Accelerated MRI Datacited by 340
  9. 2017Stable Architectures for Deep Neural Networkscited by 340
  10. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  11. 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
  12. 2015Training Very Deep Networkscited by 172
16 more
  1. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 139
  2. 2017Reversible Architectures for Arbitrarily Deep Residual Neural Networkscited by 69
  3. 2016AdaNet: Adaptive Structural Learning of Artificial Neural Networkscited by 54
  4. 2018Deep Residual Learning for Accelerated MRI using Magnitude and Phase Networkscited by 53
  5. 2020Stable ResNetcited by 49
  6. 2018Layer-Parallel Training of Deep Residual Neural Networkscited by 35
  7. 2019Deep learning as optimal control problems: models and numerical methodscited by 33
  8. 2018Missing Data Reconstruction in Remote Sensing image with a Unified Spatial-Temporal-Spectral Deep Convolutional Neural Networkcited by 26
  9. 2018Short-term Load Forecasting with Deep Residual Networkscited by 22
  10. 2020Fractional Deep Neural Network via Constrained Optimizationcited by 13
  11. 2021Analytical bounds on the local Lipschitz constants of ReLU networkscited by 12
  12. 2021Novel Deep neural networks for solving Bayesian statistical inversecited by 10
  13. 2019Selection dynamics for deep neural networkscited by 6
  14. 2021Deep Neural Nets with Fixed Bias Configurationcited by 4
  15. 2019Optimal Control of Parameterized Maxwell's System: Reduced Basis, Convergence Analysis, and A Posteriori Error Estimatescited by 1
  16. 2021A unified framework for optimal control of fractional in time subdiffusive semilinear PDEscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2023Lecture Notes: Neural Network Architecturescited by 2
  2. 2023Time Regularization in Optimal Time Variable Learningcited by 0
  3. 2026Layerwise goal-oriented adaptivity for neural ODEs: an optimal control perspectivecited by 0