mlsanityData to 7 Oct 2026

Utilizing Evolution Strategies to Train Transformers in Reinforcement Learning

arXiv 2501.13883 ↗First posted 23 Jan 2025cs.LG cs.NEcited by 2

Not on the map. Why?

Abstract (arXiv)

We explore the capability of evolution strategies to train an agent with a policy based on a transformer architecture in a reinforcement learning setting. We performed experiments using OpenAI's highly parallelizable evolution strategy to train Decision Transformer in the MuJoCo Humanoid locomotion environment and in the environment of Atari games, testing the ability of this black-box optimization technique to train even such relatively large and complicated models (compared to those previously tested in the literature). The examined evolution strategy proved to be, in general, capable of achieving strong results and managed to produce high-performing agents, showcasing evolution's ability to tackle the training of even such complex models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

18 of 33 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2016OpenAI Gymcited by 1,684
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  9. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  10. 2022Online Decision Transformercited by 194
  11. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 148
  12. 2017ES Is More Than Just a Traditional Finite-Difference Approximatorcited by 24
6 more
  1. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  2. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 20
  3. 2019Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimizationcited by 15
  4. 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
  5. 2022A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learningcited by 11
  6. 2018First-order and second-order variants of the gradient descent in a unified frameworkcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  2. 2025Utilizing Novelty-based Evolution Strategies to Train Transformers in Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Multi-Game Decision Transformers
  2. 2025Merge and Conquer: Evolutionarily Optimizing AI for 2048
  3. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
  4. 2023BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimization
  5. 2026Self-Evolving Embodied Agents via Skill-Harness Evolution
  6. 2026Self-Evolving AI for Humanoids: Mechanisms, Safety, and Evaluation of Post-Deployment Self-Improvement
  7. 2026RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms
  8. 2026Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner