# Learn2Hop: Learned Optimization on Rough Landscapes

arXiv: https://arxiv.org/abs/2107.09661 · HTML: https://mlsanity.com/paper/2107.09661/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 60 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [Learned Optimizers that Scale and Generalize](https://mlsanity.com/paper/1703.04813/) (2017)
- [Understanding and correcting pathologies in the training of learned optimizers](https://mlsanity.com/paper/1810.10180/) (2018)
- [Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves](https://mlsanity.com/paper/2009.11243/) (2020)
- [Learning Gradient Descent: Better Generalization and Longer Horizons](https://mlsanity.com/paper/1703.03633/) (2017)
- [Reverse engineering learned optimizers reveals known and novel mechanisms](https://mlsanity.com/paper/2011.02159/) (2020)
- [Using learned optimizers to make models robust to input noise](https://mlsanity.com/paper/1906.03367/) (2019)
- [JAX, M.D.: A Framework for Differentiable Physics](https://mlsanity.com/paper/1912.04232/) (2019)

## Built on by (most cited first, 12 of 12)

- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [VeLO: Training Versatile Learned Optimizers by Scaling Up](https://mlsanity.com/paper/2211.09760/) (2022)
- [Discovering Evolution Strategies via Meta-Black-Box Optimization](https://mlsanity.com/paper/2211.11260/) (2022)
- [evosax: JAX-based Evolution Strategies](https://mlsanity.com/paper/2212.04180/) (2022)
- [Practical tradeoffs between memory, compute, and performance in learned optimizers](https://mlsanity.com/paper/2203.11860/) (2022)
- [Transformer-Based Learned Optimization](https://mlsanity.com/paper/2212.01055/) (2022)
- [Learning to Generalize Provably in Learning to Optimize](https://mlsanity.com/paper/2302.11085/) (2023)
- [AdsorbDiff: Adsorbate Placement via Conditional Denoising Diffusion](https://mlsanity.com/paper/2405.03962/) (2024)
- [StriderNET: A Graph Reinforcement Learning Approach to Optimize Atomic Structures on Rough Energy Landscapes](https://mlsanity.com/paper/2301.12477/) (2023)
- [Celo: Training Versatile Learned Optimizers on a Compute Diet](https://mlsanity.com/paper/2501.12670/) (2025)
- [Distributional Reinforcement Learning for Scheduling of Chemical Production Processes](https://mlsanity.com/paper/2203.00636/) (2022)
- [MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures](https://mlsanity.com/paper/2506.04195/) (2025)

## Abstract (arXiv)

Optimization of non-convex loss surfaces containing many local minima remains a critical problem in a variety of domains, including operations research, informatics, and material design. Yet, current techniques either require extremely high iteration counts or a large number of random restarts for good performance. In this work, we propose adapting recent developments in meta-learning to these many-minima problems by learning the optimization algorithm for various loss landscapes. We focus on problems from atomic structural optimization--finding low energy configurations of many-atom systems--including widely studied models such as bimetallic clusters and disordered silicon. We find that our optimizer learns a 'hopping' behavior which enables efficient exploration and improves the rate of low energy minima discovery. Finally, our learned optimizers show promising generalization with efficiency gains on never before seen tasks (e.g. new elements or compositions). Code will be made available shortly.