Forecaster: Towards Temporally Abstract Tree-Search Planning from Pixels
On the map: World models → Latent predictive world models
Abstract (arXiv)
The ability to plan at many different levels of abstraction enables agents to envision the long-term repercussions of their decisions and thus enables sample-efficient learning. This becomes particularly beneficial in complex environments from high-dimensional state space such as pixels, where the goal is distant and the reward sparse. We introduce Forecaster, a deep hierarchical reinforcement learning approach which plans over high-level goals leveraging a temporally abstract world model. Forecaster learns an abstract model of its environment by modelling the transitions dynamics at an abstract level and training a world model on such transition. It then uses this world model to choose optimal high-level goals through a tree-search planning procedure. It additionally trains a low-level policy that learns to reach those goals. Our method not only captures building world models with longer horizons, but also, planning with such models in downstream tasks. We empirically demonstrate Forecaster's potential in both single-task learning and generalization to new tasks in the AntMaze domain.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 7 of its 12 references that are in the map. First posted on arXiv 16 Oct 2023.
Builds on
7 of 12 references.
Closest ancestors · 2
Same direction, newest first
- 2022Deep Hierarchical Planning from Pixels8 Jun
- 2018Learning Latent Dynamics for Planning from Pixels12 Nov
Other references · 5
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
- 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
- 2021Hierarchical Skills for Efficient Explorationcited by 38
- 2021Temporally Abstract Partial Modelscited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2020Evolutionary Planning in Latent Space
- 2025Spatiotemporal Forecasting as Planning: A Model-Based Reinforcement Learning Approach with Generative World Models
- 2025Enter the Void - Planning to Seek Entropy When Reward is Scarce
- 2020World Model as a Graph: Learning Latent Landmarks for Planning
- 2026Agentic Transformers Provably Learn to Search via Reinforcement Learning
- 2026Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents
- 2025Reinforcement Learning with Anticipation: A Hierarchical Approach for Long-Horizon Tasks
- 2025Learning with Expert Abstractions for Efficient Multi-Task Continuous Control