mlsanityData to 14 Oct 2026

Curiosity-Driven Imagination: Discovering Plan Operators and Learning Associated Policies for Open-World Adaptation

arXiv 2503.04931 ↗First posted 6 Mar 2025cs.RO cs.AIcited by 3

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Adapting quickly to dynamic, uncertain environments-often called "open worlds"-remains a major challenge in robotics. Traditional Task and Motion Planning (TAMP) approaches struggle to cope with unforeseen changes, are data-inefficient when adapting, and do not leverage world models during learning. We address this issue with a hybrid planning and learning system that integrates two models: a low level neural network based model that learns stochastic transitions and drives exploration via an Intrinsic Curiosity Module (ICM), and a high level symbolic planning model that captures abstract transitions using operators, enabling the agent to plan in an "imaginary" space and generate reward machines. Our evaluation in a robotic manipulation domain with sequential novelty injections demonstrates that our approach converges faster and outperforms state-of-the-art hybrid methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 40 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

23 of 40 references.

References · 23

  1. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  2. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  3. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  4. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  5. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 114
  6. 2017Environment-Independent Task Specifications via GLTLcited by 67
  7. 2021Learning Symbolic Operators for Task and Motion Planningcited by 58
  8. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 58
  9. 2018PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcited by 52
  10. 2021Learning Neuro-Symbolic Relational Transition Models for Bilevel Planningcited by 43
  11. 2020Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learningcited by 35
  12. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 34
11 more
  1. 2023Neurosymbolic Reinforcement Learning and Planning: A Surveycited by 24
  2. 2020SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learningcited by 23
  3. 2022AI Autonomy : Self-Initiated Open-World Continual Learning and Adaptationcited by 23
  4. 2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
  5. 2021Reinforcement Learning for Classical Planning: Viewing Heuristics as Dense Reward Generatorscited by 21
  6. 2023Neuro-Symbolic World Models for Adapting to Open World Noveltycited by 14
  7. 2022RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environmentscited by 13
  8. 2022Differential Assessment of Black-Box AI Agentscited by 6
  9. 2020DREAM Architecture: a Developmental Approach to Open-Ended Learning in Roboticscited by 5
  10. 2020Flexible and Efficient Long-Range Planning Through Curious Explorationcited by 4
  11. 2022A Framework for Following Temporal Logic Instructions with Unknown Causal Dependenciescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Few-Shot Neuro-Symbolic Imitation Learning for Long-Horizon Planning and Actingcited by 2
  2. 2026The Price Is Not Right: Neuro-Symbolic Methods Outperform VLAs on Structured Long-Horizon Manipulation Tasks with Significantly Lower Energy Consumptioncited by 1
  3. 2026Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics
  2. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation
  3. 2023Learning to Navigate from Scratch using World Models and Curiosity: the Good, the Bad, and the Ugly
  4. 2024NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds
  5. 2022Choreographer: Learning and Adapting Skills in Imagination
  6. 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data
  7. 2024Open-World Reinforcement Learning over Long Short-Term Imagination
  8. 2026World Model for Robot Learning: A Comprehensive Survey