# Solving Functional Optimization with Deep Networks and Variational Principles

arXiv: https://arxiv.org/abs/2410.06277 · HTML: https://mlsanity.com/paper/2410.06277/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 10 of 10)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Optimal control of PDEs using physics-informed neural networks](https://mlsanity.com/paper/2111.09880/) (2021)
- [Learning Riemannian Manifolds for Geodesic Motion Skills](https://mlsanity.com/paper/2106.04315/) (2021)
- [AONN: An adjoint-oriented neural network method for all-at-once solutions of parametric optimal control problems](https://mlsanity.com/paper/2302.02076/) (2023)
- [The Theory of Connections. Connecting Points](https://mlsanity.com/paper/1702.06862/) (2017)
- [Neural Time-Reversed Generalized Riccati Equation](https://mlsanity.com/paper/2312.09310/) (2023)

## Abstract (arXiv)

Can neural networks solve math problems using first a principle alone? This paper shows how to leverage the fundamental theorem of the calculus of variations to design deep neural networks to solve functional optimization without requiring training data (e.g., ground-truth optimal solutions). Our approach is particularly crucial when the solution is a function defined over an unknown interval or supporttextemdash such as in minimum-time control problems. By incorporating the necessary conditions satisfied by the optimal function solution, as derived from the calculus of variation, in the design of the deep architecture, CalVNet leverages overparameterized neural networks to learn these optimal functions directly. We validate CalVNet by showing that, without relying on ground-truth data and simply incorporating first principles, it successfully derives the Kalman filter for linear filtering, the bang-bang optimal control for minimum-time problems, and finds geodesics on manifolds. Our results demonstrate that CalVNet can be trained in an unsupervised manner, without relying on ground-truth data, establishing a promising framework for addressing general, potentially unsolved functional optimization problems that still lack analytical solutions.