mlsanityData to 7 Oct 2026

TorchBeast: A PyTorch Platform for Distributed RL

arXiv 1910.03552 ↗First posted 8 Oct 2019cs.LG stat.MLcited by 41

Not on the map. Why?

Abstract (arXiv)

TorchBeast is a platform for reinforcement learning (RL) research in PyTorch. It implements a version of the popular IMPALA algorithm for fast, asynchronous, parallel training of RL agents. Additionally, TorchBeast has simplicity as an explicit design goal: We provide both a pure-Python implementation ("MonoBeast") as well as a multi-machine high-performance version ("PolyBeast"). In the latter, parts of the implementation are written in C++, but all parts pertaining to machine learning are kept in simple Python using PyTorch, with the environments provided using the OpenAI Gym interface. This enables researchers to conduct scalable RL research using TorchBeast without any programming knowledge beyond Python and PyTorch. In this paper, we describe the TorchBeast design principles and implementation and demonstrate that it performs on-par with IMPALA on Atari. TorchBeast is released as an open-source package under the Apache 2.0 license and is available at urlhttps://github.com/facebookresearch/torchbeast.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 11; 2023: 8; 2024: 4; 2025: 2; 2026: 2.

41 papers.

Sort:

By area:

  1. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 312
  2. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  3. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  4. 2021Off-Belief Learningcited by 72
  5. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  6. 2022Exploration via Elliptical Episodic Bonusescited by 51
  7. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  8. 2022A Modern Self-Referential Weight Matrix That Learns to Modify Itselfcited by 40
  9. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  10. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
  11. 2022Insights From the NeurIPS 2021 NetHack Challengecited by 20
  12. 2023Exploring the Promise and Limits of Real-Time Recurrent Learningcited by 19
29 more
  1. 2021Large Batch Simulation for Deep Reinforcement Learningcited by 18
  2. 2022AutoShard: Automated Embedding Table Sharding for Recommender Systemscited by 17
  3. 2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
  4. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learningcited by 17
  5. 2021SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmarkcited by 10
  6. 2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
  7. 2022Improving Policy Learning via Language Dynamics Distillationcited by 9
  8. 2023Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learningcited by 9
  9. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 9
  10. 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
  11. 2022Hierarchical Kickstarting for Skill Transfer in Reinforcement Learningcited by 7
  12. 2023Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platformcited by 7
  13. 2025Scalable Option Learning in High-Throughput Environmentscited by 5
  14. 2022Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolboxcited by 4
  15. 2021Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precisioncited by 3
  16. 2022Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Gamescited by 3
  17. 2024SiT: Symmetry-Invariant Transformers for Generalisation in Reinforcement Learningcited by 3
  18. 2021Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learningcited by 2
  19. 2021A Novel Approach to Curiosity and Explainable Reinforcement Learning via Interpretable Sub-Goalscited by 2
  20. 2024Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstractioncited by 2
  21. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  22. 2021Guiding Global Placement With Reinforcement Learningcited by 1
  23. 2025Robust Agents in Open-Ended Worldscited by 1
  24. 2021Feudal Reinforcement Learning by Reading Manualscited by 0
  25. 2023LuckyMera: a Modular AI Framework for Building Hybrid NetHack Agentscited by 0
  26. 2023Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMOcited by 0
  27. 2023Neural architecture impact on identifying temporally extended Reinforcement Learning taskscited by 0
  28. 2026TreeTensor: Boost AI System on Nested Data with Constrained Tree-Like Tensorcited by 0
  29. 2026PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Gamescited by 0