mlsanityData to 14 Oct 2026

MRS: Multi-Resolution Skills for HRL Agents

arXiv 2505.21410 ↗First posted 27 May 2025cs.AI cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Hierarchical reinforcement learning (HRL) decomposes the policy into a manager and a worker, enabling long-horizon planning but introducing a performance gap on tasks requiring agility. We identify a root cause: in subgoal-based HRL, the manager's goal representation is typically learned without constraints on reachability or temporal distance from the current state, preventing precise local subgoal selection. We further show that the optimal subgoal distance is both task- and state-dependent: nearby subgoals enable precise control but amplify prediction noise, while distant subgoals produce smoother motion at the cost of geometric precision. We propose Multi-Resolution Skills (MRS), which learns multiple goal-prediction modules each specialized to a fixed temporal horizon, with a jointly trained meta-controller that selects among them based on the current state. MRS consistently outperforms fixed-resolution baselines and significantly reduces the performance gap between HRL and non-HRL state-of-the-art on DeepMind Control Suite, Gym-Robotics, and long-horizon AntMaze tasks. [Project page: https://sites.google.com/view/multi-res-skills/home]

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 31 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

22 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Probabilistic Subgoal Representations for Hierarchical Reinforcement learning24 Jun
  2. 2024Reconciling Spatial and Temporal Abstractions for Goal Representation18 Jan

Other references · 20

  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  2. 2023Mastering Diverse Domains through World Modelscited by 1,253
  3. 2018DeepMind Control Suitecited by 956
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  6. 2016The Option-Critic Architecturecited by 448
  7. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  8. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  9. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  10. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  11. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  12. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
8 more
  1. 2018Variational Option Discovery Algorithmscited by 122
  2. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
  3. 2022Deep Hierarchical Planning from Pixelscited by 101
  4. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 58
  5. 2018Learning Plannable Representations with Causal InfoGANcited by 57
  6. 2019Sub-policy Adaptation for Hierarchical Reinforcement Learningcited by 47
  7. 2016Strategic Attentive Writer for Learning Macro-Actionscited by 30
  8. 2021Hierarchical Reinforcement Learning with Timed Subgoalscited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraint
  2. 2025HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agents
  3. 2026Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RL
  4. 2026Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution
  5. 2025Hierarchical Reinforcement Learning in Multi-Goal Spatial Navigation with Autonomous Mobile Robots
  6. 2025DHP: Discrete Hierarchical Planning for Hierarchical Reinforcement Learning Agents
  7. 2024Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration ↗
  8. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention