Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Games
Not on the map. Why?
Abstract (arXiv)
Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and massive parallelism causes non-trivial difficulties for research and applications related to asynchronous commercial games. Here we introduce Lamarckian - an open-source platform featuring support for evolutionary reinforcement learning scalable to distributed computing resources. To improve the training speed and data efficiency, Lamarckian adopts optimized communication methods and an asynchronous evolutionary reinforcement learning workflow. To meet the demand for an asynchronous interface by commercial games and various methods, Lamarckian tailors an asynchronous Markov Decision Process interface and designs an object-oriented software architecture with decoupled modules. In comparison with the state-of-the-art RLlib, we empirically demonstrate the unique advantages of Lamarckian on benchmark tests with up to 6000 CPU cores: i) both the sampling efficiency and training speed are doubled when running PPO on Google football game; ii) the training speed is 13 times faster when running PBT+PPO on Pong game. Moreover, we also present two use cases: i) how Lamarckian is applied to generating behavior-diverse game AI; ii) how Lamarckian is applied to game balancing tests for an asynchronous commercial game.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 38 of its 62 references that are in the map. First posted on arXiv 21 Sep 2022.
Builds on
38 of 62 references.
References · 38
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2016OpenAI Gymcited by 1,684
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
- 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
- 2015Illuminating search spaces by mapping elitescited by 406
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
- 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
26 more
- 2017Population Based Training of Neural Networkscited by 351
- 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
- 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
- 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
- 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
- 2020Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learningcited by 139
- 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
- 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 96
- 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 78
- 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
- 2016DeepMind Labcited by 74
- 2019Emergent Coordination Through Competitioncited by 57
- 2018CEM-RL: Combining evolutionary and gradient-based methods for policy searchcited by 53
- 2018ViZDoom Competitions: Playing Doom from Pixelscited by 50
- 2021Evolving Reinforcement Learning Algorithmscited by 48
- 2019TorchBeast: A PyTorch Platform for Distributed RLcited by 41
- 2019Collaborative Evolutionary Reinforcement Learningcited by 36
- 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
- 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
- 2019Creating Pro-Level AI for a Real-Time Fighting Game Using Deep Reinforcement Learningcited by 14
- 2019Efficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learningcited by 11
- 2020TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learningcited by 11
- 2017TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlowcited by 9
- 2019Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligencecited by 9
- 2019Evolving the Hearthstone Metacited by 8
- 2018Macro action selection with deep reinforcement learning in StarCraftcited by 6
Built on by
Citing papers per year: 2023: 3.
3 papers.