mlsanityData to 7 Oct 2026

Megaverse: Simulating Embodied Agents at One Million Experiences per Second

arXiv 2107.08170 ↗First posted 17 Jul 2021cs.LG cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

We present Megaverse, a new 3D simulation platform for reinforcement learning and embodied AI research. The efficient design of our engine enables physics-based simulation with high-dimensional egocentric observations at more than 1,000,000 actions per second on a single 8-GPU node. Megaverse is up to 70x faster than DeepMind Lab in fully-shaded 3D scenes with interactive objects. We achieve this high simulation performance by leveraging batched simulation, thereby taking full advantage of the massive parallelism of modern GPUs. We use Megaverse to build a new benchmark that consists of several single-agent and multi-agent tasks covering a variety of cognitive challenges. We evaluate model-free RL on this benchmark to provide baselines and facilitate future research. The source code is available at https://www.megaverse.info

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 17 Jul 2021.

Builds on

34 of 44 references.

References · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  10. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  12. 2018On Evaluation of Embodied Navigation Agentscited by 526
22 more
  1. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  2. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  3. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  4. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  5. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  6. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  7. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  8. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  9. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
  10. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  11. 2018Episodic Curiosity through Reachabilitycited by 109
  12. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 84
  13. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 77
  14. 2017MINOS: Multimodal Indoor Simulator for Navigation in Complex Environmentscited by 66
  15. 2019An investigation of model-free planningcited by 64
  16. 2019Meta-Learning via Learned Losscited by 60
  17. 2018Neural Predictive Belief Representationscited by 39
  18. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  19. 2021Large Batch Simulation for Deep Reinforcement Learningcited by 18
  20. 2020Learning to Guide Random Searchcited by 13
  21. 2019Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputercited by 10
  22. 2020Physically Embedded Planning Problems: New Challenges for Reinforcement Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 8; 2023: 6; 2024: 1; 2025: 3.

19 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  3. 2025Robust Autonomy Emerges from Self-Playcited by 64
  4. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  5. 2022Exploration via Elliptical Episodic Bonusescited by 51
  6. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  7. 2022General Intelligence Requires Rethinking Explorationcited by 20
  8. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  9. 2022VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangementcited by 18
  10. 2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
  11. 2022TextWorldExpress: Simulating Text Games at One Million Steps Per Secondcited by 12
  12. 2021WarpDrive: Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPUcited by 10
7 more
  1. 2022IGLU Gridworld: Simple and Fast Environment for Embodied Dialog Agentscited by 9
  2. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 9
  3. 2023Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Modelscited by 5
  4. 2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Secondcited by 4
  5. 2022Many Episode Learning in a Modular Embodied Agent via End-to-End Interactioncited by 1
  6. 2025The Emergence of Complex Behavior in Large-Scale Ecological Environmentscited by 1
  7. 2024Massively Multiagent Minigames for Training Generalist Agentscited by 0