mlsanityData to 7 Oct 2026

Consistent Dropout for Policy Gradient Reinforcement Learning

arXiv 2202.11818 ↗First posted 23 Feb 2022cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Dropout has long been a staple of supervised learning, but is rarely used in reinforcement learning. We analyze why naive application of dropout is problematic for policy-gradient learning algorithms and introduce consistent dropout, a simple technique to address this instability. We demonstrate consistent dropout enables stable training with A2C and PPO in both continuous and discrete action environments across a wide range of dropout probabilities. Finally, we show that consistent dropout enables the online training of complex architectures such as GPT without needing to disable the model's native dropout.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 15 of its 22 references that are in the map. First posted on arXiv 23 Feb 2022.

Builds on

15 of 22 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  6. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  7. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  8. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  9. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  10. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  11. 2015Gradient Estimation Using Stochastic Computation Graphscited by 120
  12. 2018Generalization and Regularization in DQNcited by 117
3 more
  1. 2017RLlib: Abstractions for Distributed Reinforcement Learningcited by 53
  2. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  3. 2019Working Memory Graphscited by 23

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 3; 2024: 1; 2025: 2.

9 papers.

  1. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  2. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  3. 2023Normalization Enhances Generalization in Visual Reinforcement Learningcited by 13
  4. 2023ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantagescited by 10
  5. 2025Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Losscited by 6
  6. 2023Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesiscited by 4
  7. 2024Disentangling Uncertainty for Safe Social Navigation using Deep Reinforcement Learningcited by 2
  8. 2025Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learningcited by 1
  9. 2022Learning Control by Iterative Inversioncited by 0