# Learning solutions to some toy constrained optimization problems in infinite dimensional Hilbert spaces

arXiv: https://arxiv.org/abs/2401.01306 · HTML: https://mlsanity.com/paper/2401.01306/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 2 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models](https://mlsanity.com/paper/1707.01161/) (2017)
- [An Introduction to Image Synthesis with Generative Adversarial Nets](https://mlsanity.com/paper/1803.04469/) (2018)
- [An augmented Lagrangian deep learning method for variational problems with essential boundary conditions](https://mlsanity.com/paper/2106.14348/) (2021)
- [Long Short-term Memory RNN](https://mlsanity.com/paper/2105.06756/) (2021)
- [An Augmented Lagrangian Method for Optimization Problems in Banach Spaces](https://mlsanity.com/paper/1807.04467/) (2018)
- [Learning zeros of Fokker-Planck operators](https://mlsanity.com/paper/2306.07068/) (2023)

## Abstract (arXiv)

In this work we present deep learning implementations of two popular theoretical constrained optimization algorithms in infinite dimensional Hilbert spaces, namely, the penalty and the augmented Lagrangian methods. We test these algorithms on some toy problems originating in either calculus of variations or physics. We demonstrate that both methods are able to produce decent approximations for the test problems and are comparable in terms of different errors produced. Leveraging the common occurrence of the Lagrange multiplier update rule being computationally less expensive than solving subproblems in the penalty method, we achieve significant speedups in cases when the output of the constraint function is itself a function.