# Non-local Policy Optimization via Diversity-regularized Collaborative Exploration

arXiv: https://arxiv.org/abs/2006.07781 · HTML: https://mlsanity.com/paper/2006.07781/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2020.

## Built on by (most cited first, 13 of 13)

- [Diversity-Aware Policy Optimization for Large Language Model Reasoning](https://mlsanity.com/paper/2505.23433/) (2025)
- [Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality](https://mlsanity.com/paper/2205.13521/) (2022)
- [Discovering Diverse Nearly Optimal Policies with Successor Features](https://mlsanity.com/paper/2106.00669/) (2021)
- [Policy Space Diversity for Non-Transitive Games](https://mlsanity.com/paper/2306.16884/) (2023)
- [Diversifying AI: Towards Creative Chess with AlphaZero](https://mlsanity.com/paper/2308.09175/) (2023)
- [Guarded Policy Optimization with Imperfect Online Demonstrations](https://mlsanity.com/paper/2303.01728/) (2023)
- [The Curse of Diversity in Ensemble-Based Exploration](https://mlsanity.com/paper/2405.04342/) (2024)
- [Learning Diverse Policies with Soft Self-Generated Guidance](https://mlsanity.com/paper/2402.04539/) (2024)
- [Adaptive trajectory-constrained exploration strategy for deep reinforcement learning](https://mlsanity.com/paper/2312.16456/) (2023)
- [Diverse Policy Optimization for Structured Action Space](https://mlsanity.com/paper/2302.11917/) (2023)
- [Policy Dispersion in Non-Markovian Environment](https://mlsanity.com/paper/2302.14509/) (2023)
- [Social Interpretable Reinforcement Learning](https://mlsanity.com/paper/2401.15480/) (2024)
- [Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents](https://mlsanity.com/paper/2609.38805/) (2026)

## Abstract (arXiv)

Conventional Reinforcement Learning (RL) algorithms usually have one single agent learning to solve the task independently. As a result, the agent can only explore a limited part of the state-action space while the learned behavior is highly correlated to the agent's previous experience, making the training prone to a local minimum. In this work, we empower RL with the capability of teamwork and propose a novel non-local policy optimization framework called Diversity-regularized Collaborative Exploration (DiCE). DiCE utilizes a group of heterogeneous agents to explore the environment simultaneously and share the collected experiences. A regularization mechanism is further designed to maintain the diversity of the team and modulate the exploration. We implement the framework in both on-policy and off-policy settings and the experimental results show that DiCE can achieve substantial improvement over the baselines in the MuJoCo locomotion tasks.