mlsanityData to 7 Oct 2026

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

arXiv 2002.12292 ↗First posted 27 Feb 2020cs.LG cs.AIcited by 135

Not on the map. Why?

Abstract (arXiv)

Exploration in sparse reward environments remains one of the key challenges of model-free reinforcement learning. Instead of solely relying on extrinsic rewards provided by the environment, many state-of-the-art methods use intrinsic rewards to encourage exploration. However, we show that existing methods fall short in procedurally-generated environments where an agent is unlikely to visit a state more than once. We propose a novel type of intrinsic reward which encourages the agent to take actions that lead to significant changes in its learned state representation. We evaluate our method on multiple challenging procedurally-generated tasks in MiniGrid, as well as on tasks with high-dimensional observations used in prior work. Our experiments demonstrate that this approach is more sample efficient than existing exploration methods, particularly for procedurally-generated MiniGrid environments. Furthermore, we analyze the learned behavior as well as the intrinsic reward received by our agent. In contrast to previous approaches, our intrinsic reward does not diminish during the course of training and it rewards the agent substantially more for interacting with objects that it can control.

Key facts. Cited by 135 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 33; 2023: 28; 2024: 17; 2025: 23; 2026: 13.

135 papers.

Sort:

By area:

  1. 2025Reasoning with Exploration: An Entropy Perspectivecited by 277
  2. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  3. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  4. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  5. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  6. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  7. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  8. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  9. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  10. 2022Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexitycited by 59
  11. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  12. 2022Exploration via Elliptical Episodic Bonusescited by 51
38 more
  1. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  2. 2021Adversarially Guided Actor-Criticcited by 47
  3. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  4. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  5. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  6. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  7. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  8. 2021Learning more skills through optimistic explorationcited by 37
  9. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 36
  10. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  11. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  12. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  13. 2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
  14. 2021Decoupled Reinforcement Learning to Stabilise Intrinsically-Motivated Explorationcited by 25
  15. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  16. 2023The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agentscited by 24
  17. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  18. 2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
  19. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 21
  20. 2022General Intelligence Requires Rethinking Explorationcited by 20
  21. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  22. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  23. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  24. 2023Curious Replay for Model-based Adaptationcited by 16
  25. 2025Episodic Novelty Through Temporal Distancecited by 15
  26. 2025LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generationcited by 13
  27. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  28. 2024Potential-Based Reward Shaping For Intrinsic Motivationcited by 12
  29. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  30. 2023Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learningcited by 11
  31. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 10
  32. 2022Exploring through Random Curiosity with General Value Functionscited by 10
  33. 2023Value function estimation using conditional diffusion models for controlcited by 10
  34. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learningcited by 10
  35. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedbackcited by 10
  36. 2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learningcited by 10
  37. 2021Programmatic Reward Design by Examplecited by 9
  38. 2022A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machinescited by 9

50 most cited of 135 All 135 (JSON)