# Leveraging Multi-time Hamilton-Jacobi PDEs for Certain Scientific Machine Learning Problems

arXiv: https://arxiv.org/abs/2303.12928 · HTML: https://mlsanity.com/paper/2303.12928/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Deep transfer operator learning for partial differential equations under conditional shift](https://mlsanity.com/paper/2204.09810/) (2022)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators](https://mlsanity.com/paper/2208.11866/) (2022)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewhere](https://mlsanity.com/paper/1605.01799/) (2016)
- [Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations](https://mlsanity.com/paper/1711.03314/) (2017)
- [On Decomposition Models in Imaging Sciences and Multi-time Hamilton-Jacobi Partial Differential Equations](https://mlsanity.com/paper/1906.09502/) (2019)
- [QRnet: optimal regulator design with LQR-augmented neural networks](https://mlsanity.com/paper/2009.05686/) (2020)
- [Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs](https://mlsanity.com/paper/2105.03336/) (2021)
- [Hopf-type representation formulas and efficient algorithms for certain high-dimensional optimal control problems](https://mlsanity.com/paper/2110.02541/) (2021)
- [On Hamilton-Jacobi PDEs and image denoising models with certain non-additive noise](https://mlsanity.com/paper/2105.13997/) (2021)
- [Lax-Oleinik-type formulas and efficient algorithms for certain high-dimensional optimal control problems](https://mlsanity.com/paper/2109.14849/) (2021)

## Built on by (most cited first, 5 of 5)

- [Correcting model misspecification in physics-informed neural networks (PINNs)](https://mlsanity.com/paper/2310.10776/) (2023)
- [Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators](https://mlsanity.com/paper/2311.11262/) (2023)
- [Discovering a reaction-diffusion model for Alzheimer's disease by combining PINNs with symbolic regression](https://mlsanity.com/paper/2307.08107/) (2023)
- [A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes](https://mlsanity.com/paper/2305.03184/) (2023)
- [Leveraging Hamilton-Jacobi PDEs with time-dependent Hamiltonians for continual scientific machine learning](https://mlsanity.com/paper/2311.07790/) (2023)

## Abstract (arXiv)

Hamilton-Jacobi partial differential equations (HJ PDEs) have deep connections with a wide range of fields, including optimal control, differential games, and imaging sciences. By considering the time variable to be a higher dimensional quantity, HJ PDEs can be extended to the multi-time case. In this paper, we establish a novel theoretical connection between specific optimization problems arising in machine learning and the multi-time Hopf formula, which corresponds to a representation of the solution to certain multi-time HJ PDEs. Through this connection, we increase the interpretability of the training process of certain machine learning applications by showing that when we solve these learning problems, we also solve a multi-time HJ PDE and, by extension, its corresponding optimal control problem. As a first exploration of this connection, we develop the relation between the regularized linear regression problem and the Linear Quadratic Regulator (LQR). We then leverage our theoretical connection to adapt standard LQR solvers (namely, those based on the Riccati ordinary differential equations) to design new training approaches for machine learning. Finally, we provide some numerical examples that demonstrate the versatility and possible computational advantages of our Riccati-based approach in the context of continual learning, post-training calibration, transfer learning, and sparse dynamics identification.