# Towards Robust Learning to Optimize with Theoretical Guarantees

arXiv: https://arxiv.org/abs/2506.14263 · HTML: https://mlsanity.com/paper/2506.14263/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Online Learning Rate Adaptation with Hypergradient Descent](https://mlsanity.com/paper/1703.04782/) (2017)
- [Learned Optimizers that Scale and Generalize](https://mlsanity.com/paper/1703.04813/) (2017)
- [Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management](https://mlsanity.com/paper/1705.09412/) (2017)
- [Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis](https://mlsanity.com/paper/2007.07632/) (2020)
- [Learning Gradient Descent: Better Generalization and Longer Horizons](https://mlsanity.com/paper/1703.03633/) (2017)
- [Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems](https://mlsanity.com/paper/2006.08099/) (2020)
- [On the Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradient](https://mlsanity.com/paper/1803.06573/) (2018)
- [Safeguarded Learned Convex Optimization](https://mlsanity.com/paper/2003.01880/) (2020)
- [A Generalizable Approach to Learning Optimizers](https://mlsanity.com/paper/2106.00958/) (2021)
- [Ada-LISTA: Learned Solvers Adaptive to Varying Models](https://mlsanity.com/paper/2001.08456/) (2020)
- [Towards Constituting Mathematical Structures for Learning to Optimize](https://mlsanity.com/paper/2305.18577/) (2023)
- [Coordinated Sum-Rate Maximization in Multicell MU-MIMO with Deep Unrolling](https://mlsanity.com/paper/2202.10371/) (2022)
- [Learning to Generalize Provably in Learning to Optimize](https://mlsanity.com/paper/2302.11085/) (2023)
- [Learning to Beamform in Heterogeneous Massive MIMO Networks](https://mlsanity.com/paper/2011.03971/) (2020)

## Abstract (arXiv)

Learning to optimize (L2O) is an emerging technique to solve mathematical optimization problems with learning-based methods. Although with great success in many real-world scenarios such as wireless communications, computer networks, and electronic design, existing L2O works lack theoretical demonstration of their performance and robustness in out-of-distribution (OOD) scenarios. We address this gap by providing comprehensive proofs. First, we prove a sufficient condition for a robust L2O model with homogeneous convergence rates over all In-Distribution (InD) instances. We assume an L2O model achieves robustness for an InD scenario. Based on our proposed methodology of aligning OOD problems to InD problems, we also demonstrate that the L2O model's convergence rate in OOD scenarios will deteriorate by an equation of the L2O model's input features. Moreover, we propose an L2O model with a concise gradient-only feature construction and a novel gradient-based history modeling method. Numerical simulation demonstrates that our proposed model outperforms the state-of-the-art baseline in both InD and OOD scenarios and achieves up to 10 × convergence speedup. The code of our method can be found from https://github.com/NetX-lab/GoMathL2O-Official.