mlsanityData to 14 Oct 2026

Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time

arXiv 2505.03668 ↗First posted 6 May 2025cs.AIcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

This paper proposes an integration of temporal logical reasoning and Partially Observable Markov Decision Processes (POMDPs) to achieve interpretable decision-making under uncertainty with macro-actions. Our method leverages a fragment of Linear Temporal Logic (LTL) based on Event Calculus (EC) to generate persistent (i.e., constant) macro-actions, which guide Monte Carlo Tree Search (MCTS)-based POMDP solvers over a time horizon, significantly reducing inference time while ensuring robust performance. Such macro-actions are learnt via Inductive Logic Programming (ILP) from a few traces of execution (belief-action pairs), thus eliminating the need for manually designed heuristics and requiring only the specification of the POMDP transition model. In the Pocman and Rocksample benchmark scenarios, our learned macro-actions demonstrate increased expressiveness and generality when compared to time-independent heuristics, indeed offering substantial computational efficiency improvements.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 13 of its 32 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

13 of 32 references.

References · 13

  1. 2016DESPOT: Online POMDP Planning with Regularizationcited by 106
  2. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 72
  3. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 70
  4. 2020Induction and Exploitation of Subgoal Automata for Reinforcement Learningcited by 26
  5. 2020MAGIC: Learning Macro-Actions for Online POMDP Planningcited by 17
  6. 2021Closing the Planning-Learning Loop with Application to Autonomous Drivingcited by 12
  7. 2020Conflict-driven Inductive Logic Programmingcited by 8
  8. 2020Manipulation of Articulated Objects using Dual-arm Robots via Answer Set Programmingcited by 7
  9. 2023Learning Logic Specifications for Soft Policy Guidance in POMCPcited by 7
  10. 2024Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approachcited by 7
  11. 2022Deliberation in autonomous robotic surgery: a framework for handling anatomical uncertaintycited by 6
  12. 2025Online inductive learning from answer sets for efficient reinforcement learning explorationcited by 4
1 more
  1. 2019Macro Action Reinforcement Learning with Sequence Disentanglement using Variational Autoencodercited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Sample-Efficient Neurosymbolic Deep Reinforcement Learningcited by 1
  2. 2026Sample-efficient Neuro-symbolic Proximal Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Hindsight is Only 50/50: Unsuitability of MDP based Approximate POMDP Solvers for Multi-resolution Information Gathering
  2. 2025Robust Finite-Memory Policy Gradients for Hidden-Model POMDPs
  3. 2025Sequential Monte Carlo for Policy Optimization in Continuous POMDPs
  4. 2026Probing Dec-POMDP Reasoning in Cooperative MARL
  5. 2026Safety-critical Control Under Partial Observability: Reach-Avoid POMDP meets Belief Space Control
  6. 2024Rethinking Transformers in Solving POMDPs
  7. 2026Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability
  8. 2025Integrated Control and Active Perception in POMDPs for Temporal Logic Tasks and Information Acquisition ↗