Harnessing Bounded-Support Evolution Strategies for Policy Refinement
Not on the map. Why?
Abstract (arXiv)
Improving competent robot policies with on-policy RL is often hampered by noisy, low-signal gradients. We revisit Evolution Strategies (ES) as a policy-gradient proxy and localize exploration with bounded, antithetic triangular perturbations, suitable for policy refinement. We propose Triangular-Distribution ES (TD-ES) which pairs bounded triangular noise with a centered-rank finite-difference estimator to deliver stable, parallelizable, gradient-free updates. In a two-stage pipeline - PPO pretraining followed by TD-ES refinement - this preserves early sample efficiency while enabling robust late-stage gains. Across a suite of robotic manipulation tasks, TD-ES raises success rates by 26.5% relative to PPO and greatly reduces variance, offering a simple, compute-light path to reliable refinement.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 13 Nov 2025.
Builds on
33 of 39 references.
References · 33
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2015Trust Region Policy Optimizationcited by 2,580
- 2018World Modelscited by 1,513
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
- 2017Deep Reinforcement Learning that Matterscited by 754
- 2018Learning Dexterous In-Hand Manipulationcited by 687
- 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
- 2016The CMA Evolution Strategy: A Tutorialcited by 377
- 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 295
- 2019Challenges of Real-World Reinforcement Learningcited by 291
21 more
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
- 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
- 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
- 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 165
- 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 150
- 2017Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agentscited by 107
- 2018Benchmarking Reinforcement Learning Algorithms on Real-World Robotscited by 65
- 2018CEM-RL: Combining evolutionary and gradient-based methods for policy searchcited by 58
- 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
- 2023Evolutionary Reinforcement Learning: A Surveycited by 29
- 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 28
- 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 23
- 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
- 2024So You Think You Can Scale Up Autonomous Robot Data Collection?cited by 17
- 2019Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimizationcited by 15
- 2018Evolving Multimodal Robot Behavior via Many Stepping Stones with the Combinatorial Multi-Objective Evolutionary Algorithmcited by 5
- 2024Evolutionary Reinforcement Learning: A Systematic Review and Future Directionscited by 4
- 2025Evolutionary Policy Optimizationcited by 4
- 2024Solving Deep Reinforcement Learning Tasks with Evolution Strategies and Linear Policy Networkscited by 2
- 2024CMA-ES for Safe Optimizationcited by 2
- 2018Parameterless Stochastic Natural Gradient Method for Discrete Optimization and its Application to Hyper-Parameter Optimization for Neural Networkcited by 1
Built on by
Not cited yet.