mlsanityData to 7 Oct 2026

Parallelized Reverse Curriculum Generation

arXiv 2108.02128 ↗First posted 4 Aug 2021cs.LG cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

For reinforcement learning (RL), it is challenging for an agent to master a task that requires a specific series of actions due to sparse rewards. To solve this problem, reverse curriculum generation (RCG) provides a reverse expansion approach that automatically generates a curriculum for the agent to learn. More specifically, RCG adapts the initial state distribution from the neighborhood of a goal to a distance as training proceeds. However, the initial state distribution generated for each iteration might be biased, thus making the policy overfit or slowing down the reverse expansion rate. While training RCG for actor-critic (AC) based RL algorithms, this poor generalization and slow convergence might be induced by the tight coupling between an AC pair. Therefore, we propose a parallelized approach that simultaneously trains multiple AC pairs and periodically exchanges their critics. We empirically demonstrate that this proposed approach can improve RCG in performance and convergence, and it can also be applied to other AC based RL algorithms with adapted initial state distribution.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 4 Aug 2021.

Builds on

24 of 31 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2017Hindsight Experience Replaycited by 1,096
  7. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  8. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  9. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  10. 2016Unrolled Generative Adversarial Networkscited by 208
  11. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  12. 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 199
12 more
  1. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  2. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  3. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  4. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
  5. 2015Massively Parallel Methods for Deep Reinforcement Learningcited by 80
  6. 2017Data-efficient Deep Reinforcement Learning for Dexterous Manipulationcited by 61
  7. 2017Dual Discriminator Generative Adversarial Netscited by 55
  8. 2016Connecting Generative Adversarial Networks and Actor-Critic Methodscited by 49
  9. 2016Stacked Generative Adversarial Networkscited by 44
  10. 2018Evolutionary Generative Adversarial Networkscited by 42
  11. 2015Distributed Deep Q-Learningcited by 8
  12. 2016Generative Adversarial Parallelizationcited by 3

Built on by

Not cited yet.