mlsanityData to 14 Oct 2026

Data Augmentation for Continual RL via Adversarial Gradient Episodic Memory

arXiv 2408.13452 ↗First posted 24 Aug 2024cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Data efficiency of learning, which plays a key role in the Reinforcement Learning (RL) training process, becomes even more important in continual RL with sequential environments. In continual RL, the learner interacts with non-stationary, sequential tasks and is required to learn new tasks without forgetting previous knowledge. However, there is little work on implementing data augmentation for continual RL. In this paper, we investigate the efficacy of data augmentation for continual RL. Specifically, we provide benchmarking data augmentations for continual RL, by (1) summarising existing data augmentation methods and (2) including a new augmentation method for continual RL: Adversarial Augmentation with Gradient Episodic Memory (Adv-GEM). Extensive experiments show that data augmentations, such as random amplitude scaling, state-switch, mixup, adversarial augmentation, and Adv-GEM, can improve existing continual RL algorithms in terms of their average performance, catastrophic forgetting, and forward transfer, on robot control tasks. All data augmentation methods are implemented as plug-in modules for trivial integration into continual RL methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 40 references that are in the map. First posted on arXiv 24 Aug 2024.

Builds on

30 of 40 references.

References · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  10. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
18 more
  1. 2020Reinforcement Learning with Augmented Datacited by 468
  2. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  3. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  4. 2021Gradient Projection Memory for Continual Learningcited by 348
  5. 2017Variational Continual Learningcited by 339
  6. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  7. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  8. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  9. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  10. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  11. 2017Learning how to Active Learn: A Deep Reinforcement Learning Approachcited by 79
  12. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  13. 2020MixKD: Towards Efficient Distillation of Large-scale Language Modelscited by 65
  14. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  15. 2022Improving Task-free Continual Learning by Distributionally Robust Memory Evolutioncited by 31
  16. 2021Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentationcited by 15
  17. 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learningcited by 6
  18. 2024Tiny Refinements Elicit Resilience: Toward Efficient Prefix-Model Against LLM Red-Teamingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Continual Reinforcement Learning
  2. 2026Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?
  3. 2026Don't Forget the Critic: Value-Based Data Rehearsal for Multi-Cyclic Continual Reinforcement Learning
  4. 2026An Analysis of Streaming Deep Reinforcement Learning for Adaptive Continual Learning in Robotics
  5. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
  6. 2024Reinforcement Learning with Euclidean Data Augmentation for State-Based Continuous Control
  7. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  8. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learning