mlsanityData to 14 Oct 2026

Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning

arXiv 2608.02993 ↗First posted 4 Aug 2026cs.AI cs.ROcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

(Flat) Reinforcement Learning (RL) agents face significant challenges in environments with sparse rewards that require long-horizon reasoning. A compelling approach to improve sample efficiency is to incorporate knowledge into learning and decision-making. In standard Hierarchical RL (HRL), knowledge is encoded in a fixed, non-updatable form, such as architectural choices, and remains unchanged throughout learning. With fixed HRL, reasoning with incremental knowledge learned during exploration is impractical before sufficient environmental knowledge is acquired, leading to poor sample efficiency. In this work, we propose neurosymbolic HRL with em Incremental Knowledge (InK): symbolic high-level components perform em symbolic planning (e.g. using D^*) on an updatable representation of current InK, while low-level goal-conditioned neural modules learn motion primitives through experience using reward shaping. Experiments on navigation tasks demonstrate that incorporating InK substantially improves sample efficiency. Additionally, to perform em optimal symbolic planning given em prior knowledge about the world, we develop Belief World Tree Search. The code is available at https://github.com/CPS-research-group/inkbwts.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 47 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

26 of 47 references.

References · 26

  1. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  2. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  3. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  4. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  5. 2018Semi-parametric Topological Memory for Navigationcited by 209
  6. 2016Learning to Navigate in Complex Environmentscited by 198
  7. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  8. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 150
  9. 2019Planning with Goal-Conditioned Policiescited by 149
  10. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 78
  11. 2017PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planningcited by 69
  12. 2019Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?cited by 68
14 more
  1. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 47
  2. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
  3. 2023Neurosymbolic Reinforcement Learning and Planning: A Surveycited by 24
  4. 2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
  5. 2019RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators from RL Policiescited by 20
  6. 2019Learned Critical Probabilistic Roadmaps for Robotic Motion Planningcited by 18
  7. 2021Active Learning of Abstract Plan Feasibilitycited by 11
  8. 2021Map Induction: Compositional spatial submap learning for efficient exploration in novel environmentscited by 9
  9. 2023PDSketch: Integrated Planning Domain Programming and Learningcited by 7
  10. 2024Reconciling Spatial and Temporal Abstractions for Goal Representationcited by 6
  11. 2024Task and Motion Planning for Execution in the Realcited by 6
  12. 2022Combining Planning, Reasoning and Reinforcement Learning to solve Industrial Robot Taskscited by 4
  13. 2022Towards an Interpretable Hierarchical Agent Framework using Semantic Goalscited by 3
  14. 2024Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Mapscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sample-Efficient Neurosymbolic Deep Reinforcement Learning
  2. 2023Goal Space Abstraction in Hierarchical Reinforcement Learning via Reachability Analysis
  3. 2025Hierarchical Neuro-Symbolic Decision Transformer
  4. 2026Boosting deep Reinforcement Learning using pretraining with Logical Options
  5. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
  6. 2025Beyond Policy Optimization: A Data Curation Flywheel for Sparse-Reward Long-Horizon Planning
  7. 2023Goal Space Abstraction in Hierarchical Reinforcement Learning via Set-Based Reachability Analysis
  8. 2026Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers