# Time Regularization in Optimal Time Variable Learning

arXiv: https://arxiv.org/abs/2306.16111 · HTML: https://mlsanity.com/paper/2306.16111/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 28 Jun 2023.

## Builds on (most cited first, 12 of 12)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [Deep learning as optimal control problems: models and numerical methods](https://mlsanity.com/paper/1904.05657/) (2019)
- [Fractional Deep Neural Network via Constrained Optimization](https://mlsanity.com/paper/2004.00719/) (2020)
- [Novel Deep neural networks for solving Bayesian statistical inverse](https://mlsanity.com/paper/2102.03974/) (2021)
- [Selection dynamics for deep neural networks](https://mlsanity.com/paper/1905.09076/) (2019)
- [Learning across scales - A multiscale method for Convolution Neural Networks](https://mlsanity.com/paper/1703.02009/) (2017)
- [An Optimal Time Variable Learning Framework for Deep Neural Networks](https://mlsanity.com/paper/2204.08528/) (2022)
- [Lecture Notes: Neural Network Architectures](https://mlsanity.com/paper/2304.05133/) (2023)

## Abstract (arXiv)

Recently, optimal time variable learning in deep neural networks (DNNs) was introduced in arXiv:2204.08528. In this manuscript we extend the concept by introducing a regularization term that directly relates to the time horizon in discrete dynamical systems. Furthermore, we propose an adaptive pruning approach for Residual Neural Networks (ResNets), which reduces network complexity without compromising expressiveness, while simultaneously decreasing training time. The results are illustrated by applying the proposed concepts to classification tasks on the well known MNIST and Fashion MNIST data sets. Our PyTorch code is available on https://github.com/frederikkoehne/time_variable_learning.