# Optimization Landscapes Learned: Proxy Networks Boost Convergence in Physics-based Inverse Problems

arXiv: https://arxiv.org/abs/2501.16573 · HTML: https://mlsanity.com/paper/2501.16573/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 53 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Builds on (most cited first, 18 of 18)

- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process](https://mlsanity.com/paper/1904.09080/) (2019)
- [Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey](https://mlsanity.com/paper/1911.12116/) (2019)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Physics-informed neural networks for inverse problems in supersonic flows](https://mlsanity.com/paper/2202.11821/) (2022)
- [Why neural networks find simple solutions: the many regularizers of geometric complexity](https://mlsanity.com/paper/2209.13083/) (2022)
- [Solving inverse problems using conditional invertible neural networks](https://mlsanity.com/paper/2007.15849/) (2020)
- [Learning to Solve PDE-constrained Inverse Problems with Graph Networks](https://mlsanity.com/paper/2206.00711/) (2022)
- [Benchmarking deep inverse models over time, and the neural-adjoint method](https://mlsanity.com/paper/2009.12919/) (2020)
- [Regularization by architecture: A deep prior approach for inverse problems](https://mlsanity.com/paper/1812.03889/) (2018)
- [A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations](https://mlsanity.com/paper/1806.02957/) (2018)
- [Learning Regularization Parameters of Inverse Problems via Deep Neural Networks](https://mlsanity.com/paper/2104.06594/) (2021)
- [DeepCFD: Efficient Steady-State Laminar Flow Approximation with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2004.08826/) (2020)
- [Inverse design of two-dimensional materials with invertible neural networks](https://mlsanity.com/paper/2106.03013/) (2021)
- [An Unsupervised Approach to Solving Inverse Problems using Generative Adversarial Networks](https://mlsanity.com/paper/1805.07281/) (2018)
- [Inversion of Integral Models: a Neural Network Approach](https://mlsanity.com/paper/2105.15044/) (2021)
- [Physics-informed neural network for seismic wave inversion in layered semi-infinite domain](https://mlsanity.com/paper/2305.05150/) (2023)

## Abstract (arXiv)

Solving inverse problems in physics is central to understanding complex systems and advancing technologies in various fields. Iterative optimization algorithms, commonly used to solve these problems, often encounter local minima, chaos, or regions with zero gradients. This is due to their overreliance on local information and highly chaotic inverse loss landscapes governed by underlying partial differential equations (PDEs). In this work, we show that deep neural networks successfully replicate such complex loss landscapes through spatio-temporal trajectory inputs. They also offer the potential to control the underlying complexity of these chaotic loss landscapes during training through various regularization methods. We show that optimizing on network-smoothened loss landscapes leads to improved convergence in predicting optimum inverse parameters over conventional momentum-based optimizers such as BFGS on multiple challenging problems.