# POP: Prior-Fitted First-Order Optimization Policies

arXiv: https://arxiv.org/abs/2602.15473 · HTML: https://mlsanity.com/paper/2602.15473/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training](https://mlsanity.com/paper/2305.14342/) (2023)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [The Road Less Scheduled](https://mlsanity.com/paper/2405.15682/) (2024)
- [Improving Generalization Performance by Switching from Adam to SGD](https://mlsanity.com/paper/1712.07628/) (2017)
- [An Improved Analysis of Stochastic Gradient Descent with Momentum](https://mlsanity.com/paper/2007.07989/) (2020)
- [HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation](https://mlsanity.com/paper/2012.03826/) (2020)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves](https://mlsanity.com/paper/2009.11243/) (2020)
- [VeLO: Training Versatile Learned Optimizers by Scaling Up](https://mlsanity.com/paper/2211.09760/) (2022)
- [Learning an Adaptive Learning Rate Schedule](https://mlsanity.com/paper/1909.09712/) (2019)
- [Learning to Optimize in Swarms](https://mlsanity.com/paper/1911.03787/) (2019)
- [Meta-Learning for Black-box Optimization](https://mlsanity.com/paper/1907.06901/) (2019)
- [Can Learned Optimization Make Reinforcement Learning Less Difficult?](https://mlsanity.com/paper/2407.07082/) (2024)
- [Meta Learning Black-Box Population-Based Optimizers](https://mlsanity.com/paper/2103.03526/) (2021)
- [Learning to Optimize for Reinforcement Learning](https://mlsanity.com/paper/2302.01470/) (2023)
- [Learning to Generalize Provably in Learning to Optimize](https://mlsanity.com/paper/2302.11085/) (2023)
- [Pre-Training LLMs on a budget: A comparison of three optimizers](https://mlsanity.com/paper/2507.08472/) (2025)
- [Celo: Training Versatile Learned Optimizers on a Compute Diet](https://mlsanity.com/paper/2501.12670/) (2025)
- [Narrowing the Focus: Learned Optimizers for Pretrained Models](https://mlsanity.com/paper/2408.09310/) (2024)

## Abstract (arXiv)

Gradient-based optimizers are highly sensitive to design choices in their adaptive learning rate mechanisms. To address this limitation, we introduce POP, a meta-learned Reinforcement Learning (RL) policy that predicts adaptive learning rates for gradient descent, conditioned on the contextual information provided in the optimization trajectory. Our method introduces a novel RL reward formulation, a new function-scaling strategy for in-distribution generalization, and a novel prior that is used to sample millions of synthetic optimization problems. We evaluate POP on an established benchmark including 43 optimization functions of various complexity, where it significantly outperforms gradient-based methods. Our evaluation demonstrates strong generalization capabilities without task-specific tuning.