mlsanityData to 14 Oct 2026

Plannable Approximations to MDP Homomorphisms: Equivariance under Actions

arXiv 2002.11963 ↗First posted 27 Feb 2020cs.LG stat.MLcited by 61

Not on the map. Why?

Abstract (arXiv)

This work exploits action equivariance for representation learning in reinforcement learning. Equivariance under actions states that transitions in the input space are mirrored by equivalent transitions in latent space, while the map and transition functions should also commute. We introduce a contrastive loss function that enforces action equivariance on the learned representations. We prove that when our loss is zero, we have a homomorphism of a deterministic Markov Decision Process (MDP). Learning equivariant maps leads to structured latent spaces, allowing us to build a model on which we plan through value iteration. We show experimentally that for deterministic MDPs, the optimal policy in the abstract MDP can be successfully lifted to the original MDP. Moreover, the approach easily adapts to changes in the goal states. Empirically, we show that in such MDPs, we obtain better representations in fewer epochs compared to representation learning approaches using reconstructions, while generalizing better to new goals than model-free approaches.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 20; 2022: 11; 2023: 9; 2024: 8; 2025: 7; 2026: 5.

61 papers.

Sort:

By area:

  1. 2021Mastering Atari Games with Limited Datacited by 237
  2. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  3. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 75
  4. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  5. 2022Learning Symmetric Embeddings for Equivariant World Modelscited by 54
  6. 2021Towards Robust Bisimulation Metric Learningcited by 49
  7. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 44
  8. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 41
  9. 2023EDGI: Equivariant Diffusion for Planning with Embodied Agentscited by 36
  10. 2021Proper Value Equivalencecited by 33
  11. 2021LASER: Learning a Latent Action Space for Efficient Reinforcement Learningcited by 32
  12. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
38 more
  1. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 27
  2. 2021Neural Algorithmic Reasoners are Implicit Plannerscited by 21
  3. 2021Cross-Trajectory Representation Learning for Zero-Shot Generalization in RLcited by 20
  4. 2024Deep SE(3)-Equivariant Geometric Reasoning for Precise Placement Taskscited by 19
  5. 2022Transfer RL across Observation Feature Spaces via Model-Based Regularizationcited by 18
  6. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  7. 2021Model-based Meta Reinforcement Learning using Graph Structured Surrogate Modelscited by 17
  8. 2022Equivariant Representation Learning via Class-Pose Decompositioncited by 17
  9. 2020DeepAveragers: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPscited by 16
  10. 2023Neurosymbolic Grounding for Compositional World Modelscited by 14
  11. 2021Hierarchies of Planning and Reinforcement Learning for Robot Navigationcited by 11
  12. 2025Four Principles for Physically Interpretable World Modelscited by 10
  13. 2021Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spacescited by 9
  14. 2023Policy Gradient Methods in the Presence of Symmetries and State Abstractionscited by 9
  15. 2022Learning Value Functions from Undirected State-only Experiencecited by 8
  16. 2023What model does MuZero learn?cited by 7
  17. 2022CUP: Critic-Guided Policy Reusecited by 6
  18. 2024Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstractioncited by 6
  19. 2024Offline Reinforcement Learning from Datasets with Structured Non-Stationaritycited by 5
  20. 2025Adversarial Diffusion for Robust Reinforcement Learningcited by 4
  21. 2021Play to Grade: Testing Coding Games as Classifying Markov Decision Processcited by 3
  22. 2022Latent Policies for Adversarial Imitation Learningcited by 3
  23. 2025Deep SPI: Safe Policy Improvement via World Modelscited by 3
  24. 2021Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metricscited by 2
  25. 2021The Impact of Negative Sampling on Contrastive Structured World Modelscited by 2
  26. 2021Expert-Guided Symmetry Detection in Markov Decision Processescited by 2
  27. 2022An Analysis of Model-Based Reinforcement Learning From Abstracted Observationscited by 2
  28. 2023Discovering User Types: Mapping User Traits by Task-Specific Behaviors in Reinforcement Learningcited by 2
  29. 2024Reinforcement Learning Interventions on Boundedly Rational Human Agents in Frictionful Taskscited by 2
  30. 2026Regularized Latent Dynamics Prediction is a Strong Baseline For Behavioral Foundation Modelscited by 2
  31. 2021Augmenting Imitation Experience via Equivariant Representationscited by 1
  32. 2021Data Augmentation through Expert-guided Symmetry Detection to Improve Performance in Offline Reinforcement Learningcited by 1
  33. 2023IOB: Integrating Optimization Transfer and Behavior Transfer for Multi-Policy Reusecited by 1
  34. 2024Model approximation in MDPs with unbounded per-step costcited by 1
  35. 2024Symmetric Linear Bandits with Hidden Symmetrycited by 1
  36. 2024Generalization of Compositional Tasks with Logical Specification via Implicit Planningcited by 1
  37. 2025Symmetries-enhanced Multi-Agent Reinforcement Learningcited by 1
  38. 2025Learning Abstract World Models with a Group-Structured Latent Spacecited by 1

50 most cited of 61 All 61 (JSON)