mlsanityData to 14 Oct 2026

Autonomous state-space segmentation for Deep-RL sparse reward scenarios

arXiv 2504.03420 ↗First posted 4 Apr 2025cs.LG cs.AI cs.CVcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Dealing with environments with sparse rewards has always been crucial for systems developed to operate in autonomous open-ended learning settings. Intrinsic Motivations could be an effective way to help Deep Reinforcement Learning algorithms learn in such scenarios. In fact, intrinsic reward signals, such as novelty or curiosity, are generally adopted to improve exploration when extrinsic rewards are delayed or absent. Building on previous works, we tackle the problem of learning policies in the presence of sparse rewards by proposing a two-level architecture that alternates an ''intrinsically driven'' phase of exploration and autonomous sub-goal generation, to a phase of sparse reward, goal-directed policy learning. The idea is to build several small networks, each one specialized on a particular sub-path, and use them as starting points for future exploration without the need to further explore from scratch previously learnt paths. Two versions of the system have been trained and tested in the Gym SuperMarioBros environment without considering any additional extrinsic reward. The results show the validity of our approach and the importance of autonomously segment the environment to generate an efficient path towards the final goal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 16 references that are in the map. First posted on arXiv 4 Apr 2025.

Builds on

10 of 16 references.

References · 10

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  3. 2015Prioritized Experience Replaycited by 1,099
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  7. 2019Self-Supervised Exploration via Disagreementcited by 265
  8. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  9. 2021Discovering and Achieving Goals via World Modelscited by 131
  10. 2023A Definition of Open-Ended Learning Problems for Goal-Conditioned Agentscited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Intrinsic Reward Policy Optimization for Sparse-Reward Environments
  2. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning
  3. 2024Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning
  4. 2024Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Models
  5. 2022Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation
  6. 2026Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization
  7. 2025LLM-Driven Intrinsic Motivation for Sparse Reward Reinforcement Learning
  8. 2025Deep Reinforcement Learning with Hybrid Intrinsic Reward Model