# Random Gradient-Free Optimization in Infinite Dimensional Spaces

arXiv: https://arxiv.org/abs/2512.20566 · HTML: https://mlsanity.com/paper/2512.20566/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 23 Dec 2025.

## Builds on (most cited first, 10 of 10)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Handbook of Convergence Theorems for (Stochastic) Gradient Methods](https://mlsanity.com/paper/2301.11235/) (2023)
- [Position: Optimization in SciML Should Employ the Function Space Geometry](https://mlsanity.com/paper/2402.07318/) (2024)
- [A stochastic subspace approach to gradient-free optimization in high dimensions](https://mlsanity.com/paper/2003.02684/) (2020)
- [Neural Q-learning for solving PDEs](https://mlsanity.com/paper/2203.17128/) (2022)
- [Physics-Informed Neural Networks and Extensions](https://mlsanity.com/paper/2408.16806/) (2024)
- [Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients](https://mlsanity.com/paper/2402.05639/) (2024)
- [Statistical Learning and Inverse Problems: A Stochastic Gradient Approach](https://mlsanity.com/paper/2209.14967/) (2022)
- [Neural Actor-Critic Methods for Hamilton-Jacobi-Bellman PDEs: Asymptotic Analysis and Numerical Studies](https://mlsanity.com/paper/2507.06428/) (2025)

## Built on by (most cited first, 1 of 1)

- [Functional Gradient Descent with Adaptive Representations](https://mlsanity.com/paper/2606.16926/) (2026)

## Abstract (arXiv)

We propose a new gradient-free method for infinite-dimensional optimization in Hilbert spaces that requires only the computation of directional derivatives. Though functional optimization is often solved through finite-dimensional gradient descent over a parametrization, such as neural networks, we instead propose to leverage the functional nature of the optimization problem to enable provable guarantees. However, infinite-dimensional gradients are often hard to compute in practice, rendering na"ive functional gradient descent intractable. To overcome this limitation, our framework leverages only directional derivatives and a pre-basis for the Hilbert space, i.e., a linearly independent set whose span is dense. This resolves the tractability issue, as pre-bases are much more accessible than full orthonormal bases or reproducing kernels -- which may not even exist -- and individual directional derivatives can be computed using automatic differentiation. We showcase the use of our method to solve partial differential equations `a la physics-informed neural networks (PINNs), where it effectively enables provable convergence.