mlsanityData to 7 Oct 2026

Equivariant MuZero

arXiv 2302.04798 ↗First posted 9 Feb 2023cs.LG cs.AI stat.MLcited by 3

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Deep reinforcement learning repeatedly succeeds in closed, well-defined domains such as games (Chess, Go, StarCraft). The next frontier is real-world scenarios, where setups are numerous and varied. For this, agents need to learn the underlying rules governing the environment, so as to robustly generalise to conditions that differ from those they were trained on. Model-based reinforcement learning algorithms, such as the highly successful MuZero, aim to accomplish this by learning a world model. However, leveraging a world model has not consistently shown greater generalisation capabilities compared to model-free alternatives. In this work, we propose improving the data efficiency and generalisation capabilities of MuZero by explicitly incorporating the symmetries of the environment in its world-model architecture. We prove that, so long as the neural networks used by MuZero are equivariant to a particular symmetry group acting on the environment, the entirety of MuZero's action-selection algorithm will also be equivariant to that group. We evaluate Equivariant MuZero on procedurally-generated MiniPacman and on Chaser from the ProcGen suite: training on a set of mazes, and then testing on unseen rotated versions, demonstrating the benefits of equivariance. Further, we verify that our performance improvements hold even when only some of the components of Equivariant MuZero obey strict equivariance, which highlights the robustness of our construction.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 23 of its 36 references that are in the map. First posted on arXiv 9 Feb 2023.

Builds on

23 of 36 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Layer Normalizationcited by 5,579
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  8. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  9. 2019General E(2)-Equivariant Steerable CNNscited by 441
  10. 2016Steerable CNNscited by 298
  11. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  12. 2017Learning to Plan Chemical Synthesescited by 175
11 more
  1. 2016Value Iteration Networkscited by 158
  2. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  3. 2020Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphscited by 90
  4. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  5. 2019An investigation of model-free planningcited by 64
  6. 2020On the role of planning in model-based deep reinforcement learningcited by 57
  7. 2022Learning Symmetric Embeddings for Equivariant World Modelscited by 54
  8. 2022MuZero with Self-competition for Rate Control in VP9 Video Compressioncited by 27
  9. 2021Multi-Agent MDP Homomorphic Networkscited by 24
  10. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  11. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.

  1. 2023EDGI: Equivariant Diffusion for Planning with Embodied Agentscited by 35
  2. 2023The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Pythoncited by 29
  3. 2023Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimizationcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2023Equivariant Data Augmentation for Generalization in Offline Reinforcement Learning
  2. 2020Smaller World Models for Reinforcement Learning
  3. 2024UniZero: Generalized and Efficient Planning with Scalable Latent World Models
  4. 2025SE(3)-Equivariant Robot Learning and Control: A Tutorial Survey
  5. 2021Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamics
  6. 2023Learning World Models with Identifiable Factorization
  7. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  8. 2026Q-Learning With World Models