mlsanityData to 7 Oct 2026

Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single

arXiv 2304.11153 ↗First posted 21 Apr 2023cs.LG cs.NE stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

We propose an evolution strategies-based algorithm for estimating gradients in unrolled computation graphs, called ES-Single. Similarly to the recently-proposed Persistent Evolution Strategies (PES), ES-Single is unbiased, and overcomes chaos arising from recursive function applications by smoothing the meta-loss landscape. ES-Single samples a single perturbation per particle, that is kept fixed over the course of an inner problem (e.g., perturbations are not re-sampled for each partial unroll). Compared to PES, ES-Single is simpler to implement and has lower variance: the variance of ES-Single is constant with respect to the number of truncated unrolls, removing a key barrier in applying ES to long inner problems using short truncations. We show that ES-Single is unbiased for quadratic inner problems, and demonstrate empirically that its variance can be substantially lower than that of PES. ES-Single consistently outperforms PES on a variety of tasks, including a synthetic benchmark task, hyperparameter optimization, training recurrent neural networks, and training learned optimizers.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 43 of its 76 references that are in the map. First posted on arXiv 21 Apr 2023.

Builds on

43 of 76 references.

References · 43

  1. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  2. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  3. 2016Learning to learn by gradient descent by gradient descentcited by 616
  4. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  5. 2021Machine learning accelerated computational fluid dynamicscited by 461
  6. 2018Meta-learning with differentiable closed-form solverscited by 395
  7. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  8. 2017Population Based Training of Neural Networkscited by 351
  9. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  10. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  11. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  12. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 232
31 more
  1. 2016Hyperparameter optimization with approximate gradientcited by 231
  2. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
  3. 2017Regularizing and Optimizing LSTM Language Modelscited by 205
  4. 2021Efficient and Modular Implicit Differentiationcited by 199
  5. 2016Learning to Optimizecited by 195
  6. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  7. 2015Gradient Estimation Using Stochastic Computation Graphscited by 120
  8. 2017Online Learning Rate Adaptation with Hypergradient Descentcited by 117
  9. 2017Learned Optimizers that Scale and Generalizecited by 109
  10. 2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 104
  11. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  12. 2014Freeze-Thaw Bayesian Optimizationcited by 86
  13. 2021Gradients are Not All You Needcited by 82
  14. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  15. 2018Stochastic Hyperparameter Optimization through Hypernetworkscited by 72
  16. 2015Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameterscited by 71
  17. 2018Understanding Short-Horizon Bias in Stochastic Meta-Optimizationcited by 68
  18. 2020Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselvescited by 57
  19. 2019PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaoscited by 56
  20. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  21. 2017Unbiased Online Recurrent Optimizationcited by 46
  22. 2017Learning to Optimize Neural Netscited by 41
  23. 2017Unbiasing Truncated Backpropagation Through Timecited by 37
  24. 2018Meta-Learning Update Rules for Unsupervised Representation Learningcited by 37
  25. 2016The Generalized Reparameterization Gradientcited by 30
  26. 2022On Implicit Bias in Overparameterized Bilevel Optimizationcited by 30
  27. 2018Approximating Real-Time Recurrent Learning with Random Kronecker Factorscited by 29
  28. 2020Using a thousand optimization tasks to learn hyperparameter search strategiescited by 29
  29. 2019Optimal Kronecker-Sum Approximation of Real Time Recurrent Learningcited by 21
  30. 2016Sensitivity analysis on chaotic dynamical system by Non-Intrusive Least Square Shadowing (NILSS)cited by 6
  31. 2019A unified view of likelihood ratio and reparameterization gradients and an optimal importance sampling schemecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 2; 2026: 1.

7 papers.

  1. 2024Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmarkcited by 113
  2. 2023DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Trainingcited by 76
  3. 2025Evolution Strategies at the Hyperscalecited by 22
  4. 2024μLO: Compute-Efficient Meta-Generalization of Learned Optimizerscited by 7
  5. 2024Moco: A Learnable Meta Optimizer for Combinatorial Optimizationcited by 5
  6. 2025Forward-Only Continual Learningcited by 1
  7. 2026Efficient Long-Horizon Learning for Learned Optimizationcited by 0