mlsanityData to 7 Oct 2026

Equivariant Data Augmentation for Generalization in Offline Reinforcement Learning

arXiv 2309.07578 ↗First posted 14 Sep 2023cs.LG cs.AI cs.ROcited by 3

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We present a novel approach to address the challenge of generalization in offline reinforcement learning (RL), where the agent learns from a fixed dataset without any additional interaction with the environment. Specifically, we aim to improve the agent's ability to generalize to out-of-distribution goals. To achieve this, we propose to learn a dynamics model and check if it is equivariant with respect to a fixed type of transformation, namely translations in the state space. We then use an entropy regularizer to increase the equivariant set and augment the dataset with the resulting transformed samples. Finally, we learn a new policy offline based on the augmented dataset, with an off-the-shelf offline RL algorithm. Our experimental results demonstrate that our approach can greatly improve the test performance of the policy on the considered environments.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 30 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

23 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems2 Mar
  2. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning10 Mar
  3. 2020Critic Regularized Regression26 Jun
  4. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  6. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning19 Feb

Other references · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2016Group Equivariant Convolutional Networkscited by 1,154
  4. 2017Hindsight Experience Replaycited by 1,096
  5. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  7. 2020Reinforcement Learning with Augmented Datacited by 452
  8. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  9. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  10. 2017Data Augmentation Generative Adversarial Networkscited by 278
  11. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  12. 2018A Kernel Theory of Modern Data Augmentationcited by 106
5 more
  1. 2017Learning to Compose Domain-Specific Transformations for Data Augmentationcited by 91
  2. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  3. 2020Rotation, Translation, and Cropping for Zero-Shot Generalizationcited by 23
  4. 2021Collect & Infer -- a fresh look at data-efficient Reinforcement Learningcited by 17
  5. 2022The Benefits of Model-Based Generalization in Reinforcement Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 1.

3 papers.

  1. 2023Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learningcited by 2
  2. 2024Deep deterministic policy gradient with symmetric data augmentation for lateral attitude tracking control of a fixed-wing aircraftcited by 1
  3. 2026Generalization in offline RL: The structure is more important than the amount of pessimismcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Equivariant Offline Reinforcement Learning
  2. 2022Domain Generalization for Robust Model-Based Offline Reinforcement Learning
  3. 2024Doubly Mild Generalization for Offline Reinforcement Learning
  4. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
  5. 2023Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation
  6. 2023Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations
  7. 2025Model-Based Offline Reinforcement Learning with Adversarial Data Augmentation
  8. 2025Evaluation-Time Policy Switching for Offline Reinforcement Learning