mlsanityData to 7 Oct 2026

Diversity-Inducing Policy Gradient: Using Maximum Mean Discrepancy to Find a Set of Diverse Policies

arXiv 1906.00088 ↗First posted 31 May 2019cs.LG stat.MLcited by 34

Not on the map. Why?

Abstract (arXiv)

Standard reinforcement learning methods aim to master one way of solving a task whereas there may exist multiple near-optimal policies. Being able to identify this collection of near-optimal policies can allow a domain expert to efficiently explore the space of reasonable solutions. Unfortunately, existing approaches that quantify uncertainty over policies are not ultimately relevant to finding policies with qualitatively distinct behaviors. In this work, we formalize the difference between policies as a difference between the distribution of trajectories induced by each policy, which encourages diversity with respect to both state visitation and action choices. We derive a gradient-based optimization technique that can be combined with existing policy gradient methods to now identify diverse collections of well-performing policies. We demonstrate our approach on benchmarks and a healthcare task.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 13; 2024: 4; 2025: 3; 2026: 3.

34 papers.

Sort:

By area:

  1. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  2. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 39
  3. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  4. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  5. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  6. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  7. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  8. 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
  9. 2022Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemblecited by 18
  10. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  11. 2021Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Gamescited by 14
  12. 2021Adaptable Agent Populations via a Generative Model of Policiescited by 12
22 more
  1. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12
  2. 2025Diversity-Enhanced Reasoning for Subjective Questionscited by 9
  3. 2026SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoningcited by 9
  4. 2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
  5. 2023Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Controlcited by 5
  6. 2024Measuring Policy Distance for Multi-Agent Reinforcement Learningcited by 4
  7. 2023Iteratively Learn Diverse Strategies with State Distance Informationcited by 3
  8. 2024Learning Diverse Policies with Soft Self-Generated Guidancecited by 3
  9. 2023Diversity Through Exclusion (DTE): Niche Identification for Reinforcement Learning through Value-Decompositioncited by 2
  10. 2023Policy Diversity for Cooperative Agentscited by 2
  11. 2023Adaptive trajectory-constrained exploration strategy for deep reinforcement learningcited by 2
  12. 2022Safe adaptation in multiagent competitioncited by 1
  13. 2022Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptationcited by 1
  14. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  15. 2023Policy Dispersion in Non-Markovian Environmentcited by 0
  16. 2023Increasing Entropy to Boost Policy Gradient Performance on Personalization Taskscited by 0
  17. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0
  18. 2024Phasic Diversity Optimization for Population-Based Reinforcement Learningcited by 0
  19. 2024Preference-Guided Reinforcement Learning for Efficient Explorationcited by 0
  20. 2025Trajectory First: A Curriculum for Discovering Diverse Policiescited by 0
  21. 2026Calibrating Generative Models to Feature Distributions with MMD Finetuningcited by 0
  22. 2026Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agentscited by 0