mlsanityData to 7 Oct 2026

Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies

arXiv 2112.13835 ↗First posted 27 Dec 2021cs.LG stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Unrolled computation graphs arise in many scenarios, including training RNNs, tuning hyperparameters through unrolled optimization, and training learned optimizers. Current approaches to optimizing parameters in such computation graphs suffer from high variance gradients, bias, slow updates, or large memory usage. We introduce a method called Persistent Evolution Strategies (PES), which divides the computation graph into a series of truncated unrolls, and performs an evolution strategies-based update step after each unroll. PES eliminates bias from these truncations by accumulating correction terms over the entire sequence of unrolls. PES allows for rapid parameter updates, has low memory usage, is unbiased, and has reasonable variance characteristics. We experimentally demonstrate the advantages of PES compared to several other methods for gradient estimation on synthetic tasks, and show its applicability to training learned optimizers and tuning hyperparameters.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 44 of its 72 references that are in the map. First posted on arXiv 27 Dec 2021.

Builds on

44 of 72 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  3. 2018World Modelscited by 1,444
  4. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  5. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  6. 2016Learning to learn by gradient descent by gradient descentcited by 616
  7. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  8. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  9. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  10. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  11. 2017Population Based Training of Neural Networkscited by 351
  12. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
32 more
  1. 2017Forward and Reverse Gradient-Based Hyperparameter Optimizationcited by 232
  2. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 219
  3. 2016Learning to Optimizecited by 195
  4. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  5. 2015Gradient Estimation Using Stochastic Computation Graphscited by 120
  6. 2017Online Learning Rate Adaptation with Hypergradient Descentcited by 117
  7. 2017Learned Optimizers that Scale and Generalizecited by 109
  8. 2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 104
  9. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97
  10. 2019Generalized Inner Loop Meta-Learningcited by 92
  11. 2014Freeze-Thaw Bayesian Optimizationcited by 86
  12. 2018Evolved Policy Gradientscited by 86
  13. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  14. 2018Stochastic Hyperparameter Optimization through Hypernetworkscited by 72
  15. 2018Understanding Short-Horizon Bias in Stochastic Meta-Optimizationcited by 68
  16. 2020Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselvescited by 57
  17. 2019PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaoscited by 56
  18. 2017Unbiased Online Recurrent Optimizationcited by 46
  19. 2017Learning to Optimize Neural Netscited by 41
  20. 2017Unbiasing Truncated Backpropagation Through Timecited by 37
  21. 2018Meta-Learning Update Rules for Unsupervised Representation Learningcited by 37
  22. 2019A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networkscited by 37
  23. 2018Approximating Real-Time Recurrent Learning with Random Kronecker Factorscited by 29
  24. 2020Using a thousand optimization tasks to learn hyperparameter search strategiescited by 29
  25. 2019Adaptively Truncating Backpropagation Through Time to Control Gradient Biascited by 22
  26. 2019Optimal Kronecker-Sum Approximation of Real Time Recurrent Learningcited by 21
  27. 2020A Practical Sparse Approximation for Real Time Recurrent Learningcited by 18
  28. 2018Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networkscited by 13
  29. 2019On the Variance of Unbiased Online Recurrent Optimizationcited by 7
  30. 2019Efficient Optimization of Loops and Limits with Randomized Telescoping Sumscited by 7
  31. 2018Parallel Architecture and Hyperparameter Search via Successive Halving and Classificationcited by 5
  32. 2019MARTHE: Scheduling the Learning Rate Via Online Hypergradientscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 17; 2024: 4; 2025: 12; 2026: 2.

52 papers.

Sort:

By area:

  1. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  2. 2023Denoising Diffusion Samplerscited by 144
  3. 2023Data Distillation: A Surveycited by 86
  4. 2022Tutorial on amortized optimizationcited by 83
  5. 2022Theseus: A Library for Differentiable Nonlinear Optimizationcited by 64
  6. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  7. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
  8. 2023Dataset Distillation with Convexified Implicit Gradientscited by 46
  9. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  10. 2022evosax: JAX-based Evolution Strategiescited by 39
  11. 2023Fast Graph Condensation with Structure-based Neural Tangent Kernelcited by 36
  12. 2022Re-parameterizing Your Optimizers rather than Architecturescited by 35
38 more
  1. 2022Practical tradeoffs between memory, compute, and performance in learned optimizerscited by 32
  2. 2022A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biasescited by 30
  3. 2023Probabilistic Bilevel Coreset Selectioncited by 26
  4. 2024Universal Neural Functionalscited by 25
  5. 2025Evolution Strategies at the Hyperscalecited by 22
  6. 2023Improving physics-informed neural networks with meta-learned optimizationcited by 19
  7. 2023B2Opt: Learning to Optimize Black-box Optimization with Little Budgetcited by 15
  8. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  9. 2022Transformer-Based Learned Optimizationcited by 13
  10. 2023Learning to Optimize for Reinforcement Learningcited by 12
  11. 2025Learning Regularization Functionals for Inverse Problems: A Comparative Studycited by 12
  12. 2025Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunitiescited by 10
  13. 2022Amortized Proximal Optimizationcited by 9
  14. 2023Mnemosyne: Learning to Train Transformers with Transformerscited by 9
  15. 2023Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Singlecited by 7
  16. 2023Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithmscited by 7
  17. 2024μLO: Compute-Efficient Meta-Generalization of Learned Optimizerscited by 7
  18. 2025GradMetaNet: An Equivariant Architecture for Learning on Gradientscited by 7
  19. 2022Metrizing Fairnesscited by 6
  20. 2022Efficient Non-Parametric Optimizer Search for Diverse Taskscited by 6
  21. 2023Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategiescited by 5
  22. 2024Moco: A Learnable Meta Optimizer for Combinatorial Optimizationcited by 5
  23. 2022An Investigation of the Bias-Variance Tradeoff in Meta-Gradientscited by 4
  24. 2023M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptationcited by 4
  25. 2025Celo: Training Versatile Learned Optimizers on a Compute Dietcited by 4
  26. 2025Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimizationcited by 4
  27. 2025Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimizationcited by 4
  28. 2023Generating Transferable Adversarial Simulation Scenarios for Self-Driving via Neural Renderingcited by 3
  29. 2022Neural Simulated Annealingcited by 2
  30. 2023Federated Automatic Differentiationcited by 2
  31. 2023Meta-learning Optimizers for Communication-Efficient Learningcited by 2
  32. 2025Make Optimization Once and for All with Fine-grained Guidancecited by 1
  33. 2026Celo2: Towards Learned Optimization Free Lunchcited by 1
  34. 2022Hyper-Learning for Gradient-Based Batch Size Adaptationcited by 0
  35. 2023Communication Efficient and Privacy-Preserving Federated Learning Based on Evolution Strategiescited by 0
  36. 2025Generalized Kernel Inducing Points by Duality Gap for Dataset Distillationcited by 0
  37. 2025Variational Bayesian Pseudo-Coresetcited by 0
  38. 2025SUS backprop: linear backpropagation algorithm for long inputs in transformerscited by 0

50 most cited of 52 All 52 (JSON)