mlsanityData to 7 Oct 2026

Joint Learning of Hierarchical Neural Options and Abstract World Model

arXiv 2602.02799 ↗First posted 2 Feb 2026cs.LG cs.AIcited by 1

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Building agents that can perform new skills by composing existing skills is a long-standing goal of AI agent research. Towards this end, we investigate how to efficiently acquire a sequence of skills, formalized as hierarchical neural options. However, existing model-free hierarchical reinforcement algorithms need a lot of data. We propose a novel method, which we call AgentOWL (Option and World model Learning Agent), that jointly learns -- in a sample efficient way -- an abstract world model (abstracting across both states and time) and a set of hierarchical neural options. We show, on a subset of Object-Centric Atari games, that our method can learn more skills using less data than baseline methods and possesses learning and generalization capabilities that the baselines do not have.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 57 of its 84 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

57 of 84 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Diffusion for World Modeling: Visual Details Matter in Atari20 May
  2. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun

Other references · 55

  1. 2016Layer Normalizationcited by 5,579
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  5. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  9. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  11. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  12. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
38 more
  1. 2016The Option-Critic Architecturecited by 433
  2. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  3. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  4. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  5. 2017Noisy Networks for Explorationcited by 262
  6. 2023Reward Design with Language Modelscited by 257
  7. 2020First return, then explorecited by 256
  8. 2020The NetHack Learning Environmentcited by 196
  9. 2023Understanding plasticity in neural networkscited by 170
  10. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  11. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  12. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  13. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  14. 2018Playing hard exploration games by watching YouTubecited by 98
  15. 2022Deep Hierarchical Planning from Pixelscited by 96
  16. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  17. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  18. 2016Learning and Transfer of Modulated Locomotor Controllerscited by 59
  19. 2019Combating the Compounding-Error Problem with a Multi-step Modelcited by 52
  20. 2015Preconditioned Stochastic Gradient Descentcited by 48
  21. 2020Model-based Reinforcement Learning: A Surveycited by 39
  22. 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 38
  23. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
  24. 2024VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planningcited by 37
  25. 2023Learning adaptive planning representations with natural language guidancecited by 36
  26. 2020What can I do here? A Theory of Affordances in Reinforcement Learningcited by 33
  27. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  28. 2025PoE-World: Compositional World Modeling with Products of Programmatic Expertscited by 25
  29. 2025Code World Models for General Game Playingcited by 25
  30. 2019The Termination Criticcited by 24
  31. 2016Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replaycited by 22
  32. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 22
  33. 2024Partially Observable Task and Motion Planning with Uncertainty and Risk Awarenesscited by 17
  34. 2025Synthesizing world models for bilevel planningcited by 13
  35. 2021Temporally Abstract Partial Modelscited by 11
  36. 2024From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Modelscited by 11
  37. 2025One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Explorationcited by 9
  38. 2021Context-Specific Representation Abstraction for Deep Option Learningcited by 8

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring the limits of Hierarchical World Models in Reinforcement Learning
  2. 2021Adversarial Option-Aware Hierarchical Imitation Learning
  3. 2022Multi-Game Decision Transformers
  4. 2026CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution
  5. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learning
  6. 2026SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution
  7. 2026World Models as an Intermediary between Agents and the Real World
  8. 2023MABL: Bi-Level Latent-Variable World Model for Sample-Efficient Multi-Agent Reinforcement Learning