mlsanityData to 14 Oct 2026

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization

arXiv 2310.19668 ↗First posted 30 Oct 2023cs.LG cs.CVcited by 47

Not on the map. Why?

Cited by 32 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual reinforcement learning (RL) has shown promise in continuous control tasks. Despite its progress, current algorithms are still unsatisfactory in virtually every aspect of the performance such as sample efficiency, asymptotic performance, and their robustness to the choice of random seeds. In this paper, we identify a major shortcoming in existing visual RL methods that is the agents often exhibit sustained inactivity during early training, thereby limiting their ability to explore effectively. Expanding upon this crucial observation, we additionally unveil a significant correlation between the agents' inclination towards motorically inactive exploration and the absence of neuronal activity within their policy networks. To quantify this inactivity, we adopt dormant ratio as a metric to measure inactivity in the RL agent's network. Empirically, we also recognize that the dormant ratio can act as a standalone indicator of an agent's activity level, regardless of the received reward signals. Leveraging the aforementioned insights, we introduce DrM, a method that uses three core mechanisms to guide agents' exploration-exploitation trade-offs by actively minimizing the dormant ratio. Experiments demonstrate that DrM achieves significant improvements in sample efficiency and asymptotic performance with no broken seeds (76 seeds in total) across three continuous control benchmark environments, including DeepMind Control Suite, MetaWorld, and Adroit. Most importantly, DrM is the first model-free algorithm that consistently solves tasks in both the Dog and Manipulator domains from the DeepMind Control Suite as well as three dexterous hand manipulation tasks without demonstrations in Adroit, all based on pixel observations.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Builds on 53 of its 68 references that are in the map. First posted on arXiv 30 Oct 2023.

Builds on

53 of 68 references.

References · 53

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2023Mastering Diverse Domains through World Modelscited by 1,253
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  7. 2020Mastering Atari with Discrete World Modelscited by 1,007
  8. 2018Exploration by Random Network Distillationcited by 993
  9. 2018DeepMind Control Suitecited by 956
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  11. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  12. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
38 more
  1. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  2. 2020Reinforcement Learning with Augmented Datacited by 468
  3. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  4. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  5. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  6. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  7. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  8. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  9. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  10. 2017Noisy Networks for Explorationcited by 266
  11. 2017Count-Based Exploration with Neural Density Modelscited by 257
  12. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  13. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  14. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  15. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  16. 2019Efficient Exploration via State Marginal Matchingcited by 188
  17. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  18. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  19. 2023Understanding plasticity in neural networkscited by 174
  20. 2017Parameter Space Noise for Explorationcited by 161
  21. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  22. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  23. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  24. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  25. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  26. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  27. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  28. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  29. 2020Adaptive Reward-Free Explorationcited by 77
  30. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  31. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 57
  32. 2020On the model-based stochastic value gradient for continuous reinforcement learningcited by 47
  33. 2021UCB Momentum Q-learning: Correcting the bias without forgettingcited by 46
  34. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  35. 2022The Importance of Non-Markovianity in Maximum State Entropy Explorationcited by 35
  36. 2022Stabilizing Off-Policy Deep Reinforcement Learning from Pixelscited by 32
  37. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 28
  38. 2023Accelerating Reinforcement Learning with Value-Conditional State Entropy Explorationcited by 22

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 21; 2025: 12; 2026: 14.

47 papers.

Sort:

By area:

  1. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 38
  2. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  3. 2024A Study of Plasticity Loss in On-Policy Deep Reinforcement Learningcited by 26
  4. 2024UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AIcited by 25
  5. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  6. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 23
  7. 2024Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasetscited by 23
  8. 2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
  9. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 18
  10. 2024Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churncited by 17
  11. 2024ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularizationcited by 16
  12. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16
35 more
  1. 2025Value-Based Deep RL Scales Predictablycited by 15
  2. 2024Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusioncited by 14
  3. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  4. 2024Prioritized Generative Replaycited by 13
  5. 2025Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Frameworkcited by 12
  6. 2024Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularizationcited by 11
  7. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
  8. 2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 11
  9. 2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Losscited by 10
  10. 2024Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RLcited by 10
  11. 2025A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Controlcited by 9
  12. 2026FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoffcited by 7
  13. 2025The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learningcited by 6
  14. 2024UVIS: Unsupervised Video Instance Segmentationcited by 5
  15. 2025Revisiting Bisimulation Metric for Robust Representations in Reinforcement Learningcited by 3
  16. 2024Diverse Feature Learning by Self-distillation and Resetcited by 2
  17. 2024Higher Replay Ratio Empowers Sample-Efficient Multi-Agent Reinforcement Learningcited by 2
  18. 2024Pretrained Visual Representations in Reinforcement Learningcited by 2
  19. 2025Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulationcited by 2
  20. 2026Temporal Difference Learning with Constrained Initial Representationscited by 2
  21. 2026Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learningcited by 2
  22. 2024Hadamard Representation: Scaffolding Performance Across Model-free RLcited by 1
  23. 2025Silent Neuron Theory and Plasticity Preservation for Deep Reinforcement Learning in Adaptive Video Streamingcited by 1
  24. 2025Residual Reward Models: Leveraging Prior Knowledge for Efficient Preference-based Reinforcement Learning in Roboticscited by 1
  25. 2026Task-Aware Exploration via a Predictive Bisimulation Metriccited by 1
  26. 2026Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Explorationcited by 1
  27. 2025Multimodal Fusion for Sim2real Transfer in Visual Reinforcement Learningcited by 0
  28. 2026APC-RL: Exceeding Data-Driven Behavior Priors with Adaptive Policy Compositioncited by 0
  29. 2026Rethinking Plasticity in Deep Reinforcement Learningcited by 0
  30. 2026Autonomous Diffractometry Enabled by Visual Reinforcement Learningcited by 0
  31. 2026Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priorscited by 0
  32. 2026Reinforcement Learning for the Full Strawberry Harvesting Process: Obstacle Separation, Detachment, and Placementcited by 0
  33. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0
  34. 2026Sustaining Plasticity via Learnable Wavelet Activations in Continual Learningcited by 0
  35. 2026SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropycited by 0