mlsanityData to 7 Oct 2026

Differentiable Multiple Shooting Layers

arXiv 2106.03885 ↗First posted 7 Jun 2021cs.LG math.DS math.OCcited by 20

Not on the map. Why?

Abstract (arXiv)

We detail a novel class of implicit neural models. Leveraging time-parallel methods for differential equations, Multiple Shooting Layers (MSLs) seek solutions of initial value problems via parallelizable root-finding algorithms. MSLs broadly serve as drop-in replacements for neural ordinary differential equations (Neural ODEs) with improved efficiency in number of function evaluations (NFEs) and wall-clock inference time. We develop the algorithmic framework of MSLs, analyzing the different choices of solution methods from a theoretical and computational perspective. MSLs are showcased in long horizon optimal control of ODEs and PDEs and as latent models for sequence generation. Finally, we investigate the speedups obtained through application of MSL inference in neural controlled differential equations (Neural CDEs) for time series classification of medical data.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 19 of its 51 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

19 of 51 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2019Deep Equilibrium Modelscited by 603
  6. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  7. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  8. 2020Dissecting Neural ODEscited by 150
  9. 2019Neural Jump Stochastic Differential Equationscited by 147
  10. 2019PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEscited by 116
  11. 2018Layer-Parallel Training of Deep Residual Neural Networkscited by 35
  12. 2019DiffEqFlux.jl - A Julia Library for Neural Differential Equationscited by 33
7 more
  1. 2020Hypersolvers: Toward Fast Continuous-Depth Modelscited by 33
  2. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  3. 2020TorchDyn: A Neural Differential Equations Librarycited by 16
  4. 2020A Shooting Formulation of Deep Learningcited by 8
  5. 2020Layer-Parallel Training with GPU Concurrency of Deep Residual Neural Networks via Nonlinear Multigridcited by 8
  6. 2021Multiple-shooting adjoint method for whole-brain dynamic causal modelingcited by 5
  7. 2020A Practical Layer-Parallel Training Algorithm for Residual Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 6; 2024: 3; 2025: 1; 2026: 3.

20 papers.

Sort:

By area:

  1. 2022Monarch: Expressive Structured Matrices for Efficient and Accurate Trainingcited by 100
  2. 2023Effectively Modeling Time Series with Simple Discrete State Spacescited by 54
  3. 2023Parallelizing non-linear sequential models over the sequence lengthcited by 45
  4. 2024Towards Scalable and Stable Parallelization of Nonlinear RNNscited by 38
  5. 2022Deep Latent State Space Models for Time-Series Generationcited by 37
  6. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  7. 2024State-Free Inference of State-Space Models: The Transfer Function Approachcited by 18
  8. 2022Latent Neural ODEs with Sparse Bayesian Multiple Shootingcited by 17
  9. 2021Variational multiple shooting for Bayesian ODEs with Gaussian processescited by 15
  10. 2022Homotopy-based training of NeuralODEs for accurate dynamics discoverycited by 14
  11. 2023DeepPCR: Parallelizing Sequential Operations in Neural Networkscited by 9
  12. 2024Learning Physics Informed Neural ODEs With Partial Measurementscited by 9
8 more
  1. 2022Self-Similarity Priors: Neural Collages as Differentiable Fractal Representationscited by 6
  2. 2022Neural Solvers for Fast and Accurate Numerical Optimal Controlcited by 4
  3. 2023Hybridizing Physics and Neural ODEs for Predicting Plasma Inductance Dynamics in Tokamak Fusion Reactorscited by 4
  4. 2023Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformerscited by 1
  5. 2026Trajectory Stitching for Solving Inverse Problems with Flow-Based Modelscited by 1
  6. 2025A condensing approach to multiple shooting neural ordinary differential equationcited by 0
  7. 2026Long-Time Trajectory Approximation via SA-NODEs: Model Predictive and Floquet Strategiescited by 0
  8. 2026Neural ODEs Meet Concurrent Learning: Stable Online Learning with Lyapunov Guaranteescited by 0