# A Shooting Formulation of Deep Learning

arXiv: https://arxiv.org/abs/2006.10330 · HTML: https://mlsanity.com/paper/2006.10330/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 8 of 8)

- [Differentiable Multiple Shooting Layers](https://mlsanity.com/paper/2106.03885/) (2021)
- [Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport](https://mlsanity.com/paper/2403.12887/) (2024)
- [Variational formulations of ODE-Net as a mean-field optimal control problem and existence results](https://mlsanity.com/paper/2303.05924/) (2023)
- [Geometric Foundations of Tuning without Forgetting in Neural ODEs](https://mlsanity.com/paper/2509.03474/) (2025)
- [Imbedding Deep Neural Networks](https://mlsanity.com/paper/2202.00113/) (2022)
- [Learnable Koopman-Enhanced Transformer-Based Time Series Forecasting with Spectral Control](https://mlsanity.com/paper/2602.02592/) (2026)
- [FineMorphs: Affine-diffeomorphic sequences for regression](https://mlsanity.com/paper/2305.17255/) (2023)
- [DeepKoopFormer: A Koopman Enhanced Transformer Based Architecture for Time Series Forecasting](https://mlsanity.com/paper/2508.02616/) (2025)

## Abstract (arXiv)

Continuous-depth neural networks can be viewed as deep limits of discrete neural networks whose dynamics resemble a discretization of an ordinary differential equation (ODE). Although important steps have been taken to realize the advantages of such continuous formulations, most current techniques are not truly continuous-depth as they assume textitidentical layers. Indeed, existing works throw into relief the myriad difficulties presented by an infinite-dimensional parameter space in learning a continuous-depth neural ODE. To this end, we introduce a shooting formulation which shifts the perspective from parameterizing a network layer-by-layer to parameterizing over optimal networks described only by a set of initial conditions. For scalability, we propose a novel particle-ensemble parametrization which fully specifies the optimal weight trajectory of the continuous-depth neural network. Our experiments show that our particle-ensemble shooting formulation can achieve competitive performance, especially on long-range forecasting tasks. Finally, though the current work is inspired by continuous-depth neural networks, the particle-ensemble shooting formulation also applies to discrete-time networks and may lead to a new fertile area of research in deep learning parametrization.