mlsanityData to 14 Oct 2026

Equivariant Offline Reinforcement Learning

arXiv 2406.13961 ↗First posted 20 Jun 2024cs.LG cs.ROcited by 5

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Sample efficiency is critical when applying learning-based methods to robotic manipulation due to the high cost of collecting expert demonstrations and the challenges of on-robot policy learning through online Reinforcement Learning (RL). Offline RL addresses this issue by enabling policy learning from an offline dataset collected using any behavioral policy, regardless of its quality. However, recent advancements in offline RL have predominantly focused on learning from large datasets. Given that many robotic manipulation tasks can be formulated as rotation-symmetric problems, we investigate the use of SO(2)-equivariant neural networks for offline RL with a limited number of demonstrations. Our experimental results show that equivariant versions of Conservative Q-Learning (CQL) and Implicit Q-Learning (IQL) outperform their non-equivariant counterparts. We provide empirical evidence demonstrating how equivariance improves offline learning algorithms in the low-data regime.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 29 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

26 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulation6 Aug

Other references · 25

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2016Group Equivariant Convolutional Networkscited by 1,165
  6. 2017Constrained Policy Optimizationcited by 983
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  8. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  10. 2018Learning Dexterous In-Hand Manipulationcited by 687
  11. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  12. 2019General E(2)-Equivariant Steerable CNNscited by 446
13 more
  1. 2016Steerable CNNscited by 305
  2. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
  3. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 142
  4. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  5. 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 56
  6. 2022Sample Efficient Grasp Learning Using Equivariant Modelscited by 53
  7. 2022TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulationcited by 47
  8. 2022On-Robot Learning With Equivariant Modelscited by 42
  9. 2022The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetrycited by 33
  10. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 33
  11. 2020Group Equivariant Deep Reinforcement Learningcited by 24
  12. 2022SEIL: Simulation-augmented Equivariant Imitation Learningcited by 23
  13. 2023Leveraging Symmetries in Pick and Placecited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 1.

5 papers.

  1. 2025SE(3)-Equivariant Robot Learning and Control: A Tutorial Surveycited by 10
  2. 2025Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environmentscited by 3
  3. 2024Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrationscited by 2
  4. 2024Problem Space Transformations for Out-of-Distribution Generalisation in Behavioural Cloningcited by 0
  5. 2026Reflex: Reinforcement Learning with Reflection Symmetry Exploitation in State-Based Continuous Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Equivariant Data Augmentation for Generalization in Offline Reinforcement Learning
  2. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning
  3. 2025Equivariant Goal Conditioned Contrastive Reinforcement Learning
  4. 2021Efficient Robotic Manipulation Through Offline-to-Online Reinforcement Learning and Goal-Aware State Information
  5. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning
  6. 2022Implicit Offline Reinforcement Learning via Supervised Learning
  7. 2026Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets
  8. 2024Approximate Equivariance in Reinforcement Learning