Data to 7 Oct 2026

Parameter Space Noise for Exploration

arXiv 1706.01905 ↗First posted 6 Jun 2017cs.LG cs.AI cs.NEcited by 158

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space. An alternative is to add noise directly to the agent's parameters, which can lead to more consistent exploration and a richer set of behaviors. Methods such as evolutionary strategies use parameter perturbations, but discard all temporal structure in the process and require significantly more samples. Combining parameter noise with traditional RL methods allows to combine the best of both worlds. We demonstrate that both off- and on-policy methods benefit from this approach through experimental comparison of DQN, DDPG, and TRPO on high-dimensional discrete action environments as well as continuous control tasks. Our results show that RL with parameter noise learns more efficiently than traditional RL with action space noise and evolutionary strategies individually.

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 1; 2021: 22; 2022: 26; 2023: 29; 2024: 23; 2025: 30; 2026: 26.

158 papers.

Sort:

By area:

2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 45
2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
2022SAM as an Optimal Relaxation of Bayescited by 40
2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 40
2018Sample-Efficient Imitation Learning via Generative Adversarial Netscited by 36
2023Small batch deep reinforcement learningcited by 35
2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29

38 more

2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancementcited by 28
2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
2023Evolutionary Reinforcement Learning: A Surveycited by 24
2021Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcited by 22
2023Accelerating Reinforcement Learning with Value-Conditional State Entropy Explorationcited by 22
2022Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performancecited by 21
2023Latent Exploration for Reinforcement Learningcited by 21
2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20
2025Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learningcited by 17
2025Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returnscited by 16
2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
2021Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learningcited by 14
2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
2025QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMscited by 13
2022The Sufficiency of Off-Policyness and Soft Clipping: PPO is still Insufficient according to an Off-Policy Measurecited by 12
2023Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reusecited by 12
2024More Efficient Randomized Exploration for Reinforcement Learning via Approximate Samplingcited by 12
2021A Survey on Deep Reinforcement Learning for Data Processing and Analyticscited by 10
2021Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximationcited by 10
2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10
2023Efficient Exploration via Epistemic-Risk-Seeking Policy Optimizationcited by 10
2022Learning Robust Real-Time Cultural Transmission without Human Datacited by 9
2024Trajectory-wise Iterative Reinforcement Learning Framework for Auto-biddingcited by 9
2021Continuous Control With Ensemble Deep Deterministic Policy Gradientscited by 8
2023Maximum diffusion reinforcement learningcited by 8
2023Colored Noise in PPO: Improved Exploration and Performance through Correlated Action Samplingcited by 8
2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
2021Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learningcited by 7
2023Self-Motivated Multi-Agent Explorationcited by 7
2026Expanding LLM Agent Boundaries with Strategy-Guided Explorationcited by 7
2024Q-Star Meets Scalable Posterior Sampling: Bridging Theory and Practice via HyperAgentcited by 6
2024Random Latent Exploration for Deep Reinforcement Learningcited by 6
2021State-Aware Variational Thompson Sampling for Deep Q-Networkscited by 5

50 most cited of 158 All 158 (JSON)