mlsanityData to 7 Oct 2026

Neural Optimizer Search with Reinforcement Learning

arXiv 1709.07417 ↗First posted 21 Sep 2017cs.AI cs.LG stat.MLcited by 82

Not on the map. Why?

Abstract (arXiv)

We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a domain specific language that describes a mathematical update equation based on a list of primitive functions, such as the gradient, running average of the gradient, etc. The controller is trained with Reinforcement Learning to maximize the performance of a model after a few epochs. On CIFAR-10, our method discovers several update rules that are better than many commonly used optimizers, such as Adam, RMSProp, or SGD with and without Momentum on a ConvNet model. We introduce two new optimizers, named PowerSign and AddSign, which we show transfer well and improve training on a variety of different tasks and architectures, including ImageNet classification and Google's neural machine translation system.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 23; 2023: 10; 2024: 11; 2025: 9; 2026: 7.

82 papers.

Sort:

By area:

  1. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  2. 2021Learning to Optimize: A Primer and A Benchmarkcited by 206
  3. 2022Tutorial on amortized optimizationcited by 83
  4. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  5. 2021GLiT: Neural Architecture Search for Global and Local Image Transformercited by 53
  6. 2023Lion Secretly Solves Constrained Optimization: As Lyapunov Predictscited by 50
  7. 2021Evolving Reinforcement Learning Algorithmscited by 48
  8. 2022Automated Dynamic Algorithm Configurationcited by 37
  9. 2021AutoDropout: Learning Dropout Patterns to Regularize Deep Networkscited by 35
  10. 2022Practical tradeoffs between memory, compute, and performance in learned optimizerscited by 32
  11. 2021How to Train Your MAML to Excel in Few-Shot Classificationcited by 22
  12. 2021BN-NAS: Neural Architecture Search with Batch Normalizationcited by 21
38 more
  1. 2022Finite Expression Method for Solving High-Dimensional Partial Differential Equationscited by 21
  2. 2022Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimizationcited by 17
  3. 2022Symbolic Learning to Optimize: Towards Interpretability and Scalabilitycited by 16
  4. 2023MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReIDcited by 16
  5. 2021G-CoS: GNN-Accelerator Co-Search Towards Both Better Accuracy and Efficiencycited by 15
  6. 2023B2Opt: Learning to Optimize Black-box Optimization with Little Budgetcited by 15
  7. 2021Improving exploration in policy gradient search: Application to symbolic optimizationcited by 14
  8. 2021Training Learned Optimizers with Randomly Initialized Learned Optimizerscited by 13
  9. 2021Acceleration via Fractal Learning Rate Schedulescited by 13
  10. 2021T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imagingcited by 11
  11. 2022Selecting and Composing Learning Rate Policies for Deep Neural Networkscited by 11
  12. 2022AutoLossGen: Automatic Loss Function Generation for Recommender Systemscited by 10
  13. 2022Learning to Optimize in Model Predictive Controlcited by 10
  14. 2024Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaborationcited by 10
  15. 2026AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvementcited by 10
  16. 2021MetricOpt: Learning to Optimize Black-Box Evaluation Metricscited by 9
  17. 2022Graph-based Neural Acceleration for Nonnegative Matrix Factorizationcited by 9
  18. 2022Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammarscited by 9
  19. 2024Adaptive Non-uniform Timestep Sampling for Accelerating Diffusion Model Trainingcited by 8
  20. 2021Learning specialized activation functions with the Piecewise Linear Unitcited by 7
  21. 2022Meta Mirror Descent: Optimiser Learning for Fast Convergencecited by 7
  22. 2022Reinforcement Learning Based Query Vertex Ordering Model for Subgraph Matchingcited by 6
  23. 2022Efficient Non-Parametric Optimizer Search for Diverse Taskscited by 6
  24. 2024tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)cited by 5
  25. 2022Entangled Residual Mappingscited by 4
  26. 2023Learning To Optimize Quantum Neural Network Without Gradientscited by 4
  27. 2023DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learningcited by 4
  28. 2021Single-DARTS: Towards Stable Architecture Searchcited by 3
  29. 2021Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustnesscited by 3
  30. 2024Narrowing the Focus: Learned Optimizers for Pretrained Modelscited by 3
  31. 2025Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approachcited by 3
  32. 2021A Novel Framework for Neural Architecture Search in the Hill Climbing Domaincited by 2
  33. 2022Learning to learn online with neuromodulated synaptic plasticity in spiking neural networkscited by 2
  34. 2022EASNet: Searching Elastic and Accurate Network Architecture for Stereo Matchingcited by 2
  35. 2023Federated Automatic Differentiationcited by 2
  36. 2023Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Monthscited by 2
  37. 2024Neural Loss Function Evolution for Large-Scale Image Classifier Convolutional Neural Networkscited by 2
  38. 2024Closed-form Solutions: A New Perspective on Solving Differential Equationscited by 2

50 most cited of 82 All 82 (JSON)