mlsanityData to 7 Oct 2026

Diversity-Driven Exploration Strategy for Deep Reinforcement Learning

arXiv 1802.04564 ↗First posted 13 Feb 2018cs.AI stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

Efficient exploration remains a challenging research problem in reinforcement learning, especially when an environment contains large state spaces, deceptive local optima, or sparse rewards. To tackle this problem, we present a diversity-driven approach for exploration, which can be easily combined with both off- and on-policy reinforcement learning algorithms. We show that by simply adding a distance measure to the loss function, the proposed methodology significantly enhances an agent's exploratory behaviors, and thus preventing the policy from being trapped in local optima. We further propose an adaptive scaling method for stabilizing the learning process. Our experimental results in Atari 2600 show that our method outperforms baseline approaches in several tasks in terms of mean scores and exploration efficiency.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 13; 2023: 18; 2024: 9; 2025: 8; 2026: 5.

65 papers.

Sort:

By area:

  1. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  2. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  3. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 59
  4. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  5. 2022Winner Takes It All: Training Performant RL Populations for Combinatorial Optimizationcited by 57
  6. 2024From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactionscited by 46
  7. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  8. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 39
  9. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  10. 2021Regularized Policies are Reward Robustcited by 32
  11. 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
  12. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
38 more
  1. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  2. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 21
  3. 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
  4. 2022Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemblecited by 18
  5. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  6. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 16
  7. 2023Policy Space Diversity for Non-Transitive Gamescited by 15
  8. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  9. 2021Adaptable Agent Populations via a Generative Model of Policiescited by 12
  10. 2021Hierarchies of Planning and Reinforcement Learning for Robot Navigationcited by 11
  11. 2022Learning Robust Real-Time Cultural Transmission without Human Datacited by 9
  12. 2023Generalization through Diversity: Improving Unsupervised Environment Designcited by 9
  13. 2023Safe Deep Reinforcement Learning by Verifying Task-Level Propertiescited by 9
  14. 2025Diversity-Enhanced Reasoning for Subjective Questionscited by 9
  15. 2021Hierarchical Reinforcement Learning for Air Combat at DARPA's AlphaDogfight Trialscited by 8
  16. 2024Exclusively Penalized Q-learning for Offline Reinforcement Learningcited by 8
  17. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  18. 2025What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversitycited by 7
  19. 2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
  20. 2024Random Latent Exploration for Deep Reinforcement Learningcited by 6
  21. 2023PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Marketscited by 5
  22. 2023Robust Driving Policy Learning with Guided Meta Reinforcement Learningcited by 5
  23. 2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
  24. 2023PushWorld: A benchmark for manipulation planning with tools and movable obstaclescited by 3
  25. 2023Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policycited by 3
  26. 2024Learning Diverse Policies with Soft Self-Generated Guidancecited by 3
  27. 2024Safe Multiagent Coordination via Entropic Explorationcited by 3
  28. 2025Improving Human-AI Coordination through Online Adversarial Training and Generative Modelscited by 3
  29. 2025Learning Intractable Multimodal Policies with Reparameterization and Diversity Regularizationcited by 3
  30. 2021Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generationcited by 2
  31. 2023Diversity Through Exclusion (DTE): Niche Identification for Reinforcement Learning through Value-Decompositioncited by 2
  32. 2023Adaptive trajectory-constrained exploration strategy for deep reinforcement learningcited by 2
  33. 2022Generalized Policy Improvement Algorithms with Theoretically Supported Sample Reusecited by 1
  34. 2022Reinforcement learning with experience replay and adaptation of action dispersioncited by 1
  35. 2022Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptationcited by 1
  36. 2022CIM: Constrained Intrinsic Motivation for Sparse-Reward Continuous Controlcited by 1
  37. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  38. 2024Toward Debugging Deep Reinforcement Learning Programs with RLExplorercited by 1

50 most cited of 65 All 65 (JSON)