mlsanityData to 14 Oct 2026

Learning Multi-Timescale Abstractions for Hierarchical Combinatorial Planning

arXiv 2605.17058 ↗First posted 16 May 2026cs.LGcited by 0

On the map: World models → Latent predictive world models

also in 1 more
Abstract (arXiv)

The combination of exponentially large action spaces, stochastic dynamics, and long-horizon decision-making under limited resources makes Sequential Stochastic Combinatorial Optimization (SSCO) particularly challenging for reinforcement learning. Hierarchical Reinforcement Learning (HRL) offers a natural decomposition, but it places the high-level policy in a Semi-Markov Decision Process (SMDP) where actions have variable durations, making it difficult to learn a world model that is suitable for planning. We introduce a model-based hierarchical framework for sequential stochastic combinatorial decision-making that directly addresses this issue. Our method combines a latent-space tree-search planner with an SMDP-aware world model for variable-duration decisions. A multi-timescale objective structures the latent dynamics so that transition magnitudes reflect the effective temporal scales of abstract actions, enabling efficient lookahead under adaptive temporal abstraction. We further learn a subgoal-conditioned budget policy jointly with the world model to support context-aware resource allocation. Across challenging SSCO benchmarks, our method outperforms strong baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 23 of its 38 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

23 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Deep Hierarchical Planning from Pixels8 Jun
  2. 2020World Model as a Graph: Learning Latent Landmarks for Planning25 Nov

Other references · 21

  1. 2018Neural Ordinary Differential Equationscited by 3,779
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2015Pointer Networkscited by 895
  5. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 608
  6. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  7. 2016The Option-Critic Architecturecited by 448
  8. 2020Reinforcement Learning for Combinatorial Optimization: A Surveycited by 236
  9. 2018Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Searchcited by 183
  10. 2019Planning with Goal-Conditioned Policiescited by 149
  11. 2021Learning and Planning in Complex Action Spacescited by 77
  12. 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 58
9 more
  1. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Informationcited by 41
  2. 2020Options of Interest: Temporal Abstraction with Interest Functionscited by 30
  3. 2021Contingency-Aware Influence Maximization: A Reinforcement Learning Approachcited by 13
  4. 2022State-Conditioned Adversarial Subgoal Generationcited by 8
  5. 2023Multi Time Scale World Modelscited by 8
  6. 2024Probabilistic Subgoal Representations for Hierarchical Reinforcement learningcited by 8
  7. 2025Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoalscited by 5
  8. 2020Time-constrained Adaptive Influence Maximizationcited by 4
  9. 2025Sequential Stochastic Combinatorial Optimization Using Hierarchal Reinforcement Learningcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Structured Reinforcement Learning for Combinatorial Decision-Making
  2. 2026On Semi-Markov Suboptimality in Hierarchical Reinforcement Learning
  3. 2026Laplacian Representations for Decision-Time Planning
  4. 2026Hierarchical Reinforcement Learning with Stable Temporal Abstraction for Language Model Agents
  5. 2024Exploring the limits of Hierarchical World Models in Reinforcement Learning
  6. 2025Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstraction
  7. 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learning
  8. 2024Reinforcement Learning with Options and State Representation