mlsanityData to 7 Oct 2026

Exploiting Symmetry in Dynamics for Model-Based Reinforcement Learning with Asymmetric Rewards

arXiv 2403.19024 ↗First posted 27 Mar 2024cs.LG cs.AI cs.ROcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Recent work in reinforcement learning has leveraged symmetries in the model to improve sample efficiency in training a policy. A commonly used simplifying assumption is that the dynamics and reward both exhibit the same symmetry; however, in many real-world environments, the dynamical model exhibits symmetry independent of the reward model. In this paper, we assume only the dynamics exhibit symmetry, extending the scope of problems in reinforcement learning and learning in control theory to which symmetry techniques can be applied. We use Cartan's moving frame method to introduce a technique for learning dynamics that, by construction, exhibit specified symmetries. Numerical experiments demonstrate that the proposed method learns a more accurate dynamical model

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 10 of its 23 references that are in the map. First posted on arXiv 27 Mar 2024.

Builds on

10 of 23 references.

References · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  5. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  6. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  7. 2020Model-based Reinforcement Learning: A Surveycited by 39
  8. 2021Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamicscited by 27
  9. 2018Exploiting symmetry for discrete-time reachability computationscited by 2
  10. 2019Multi-Agent Safety Verification using Symmetry Transformationscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Symmetries-enhanced Multi-Agent Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL
  2. 2025Symmetry-Guided Multi-Agent Inverse Reinforcement Learning
  3. 2023Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning
  4. 2025Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limits
  5. 2020A Contraction Approach to Model-based Reinforcement Learning
  6. 2025Multi-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulation
  7. 2026Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation
  8. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees