# Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time

arXiv: https://arxiv.org/abs/2505.03668 · HTML: https://mlsanity.com/paper/2505.03668/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 6 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 13 of 13)

- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications](https://mlsanity.com/paper/1807.06333/) (2018)
- [Approximate information state for approximate planning and reinforcement learning in partially observed systems](https://mlsanity.com/paper/2010.08843/) (2020)
- [Induction and Exploitation of Subgoal Automata for Reinforcement Learning](https://mlsanity.com/paper/2009.03855/) (2020)
- [MAGIC: Learning Macro-Actions for Online POMDP Planning](https://mlsanity.com/paper/2011.03813/) (2020)
- [Closing the Planning-Learning Loop with Application to Autonomous Driving](https://mlsanity.com/paper/2101.03834/) (2021)
- [Conflict-driven Inductive Logic Programming](https://mlsanity.com/paper/2101.00058/) (2020)
- [Manipulation of Articulated Objects using Dual-arm Robots via Answer Set Programming](https://mlsanity.com/paper/2010.01164/) (2020)
- [Learning Logic Specifications for Soft Policy Guidance in POMCP](https://mlsanity.com/paper/2303.09172/) (2023)
- [Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach](https://mlsanity.com/paper/2402.19265/) (2024)
- [Deliberation in autonomous robotic surgery: a framework for handling anatomical uncertainty](https://mlsanity.com/paper/2203.05438/) (2022)
- [Online inductive learning from answer sets for efficient reinforcement learning exploration](https://mlsanity.com/paper/2501.07445/) (2025)
- [Macro Action Reinforcement Learning with Sequence Disentanglement using Variational Autoencoder](https://mlsanity.com/paper/1903.09366/) (2019)

## Built on by (most cited first, 2 of 2)

- [Sample-Efficient Neurosymbolic Deep Reinforcement Learning](https://mlsanity.com/paper/2601.02850/) (2026)
- [Sample-efficient Neuro-symbolic Proximal Policy Optimization](https://mlsanity.com/paper/2604.25534/) (2026)

## Abstract (arXiv)

This paper proposes an integration of temporal logical reasoning and Partially Observable Markov Decision Processes (POMDPs) to achieve interpretable decision-making under uncertainty with macro-actions. Our method leverages a fragment of Linear Temporal Logic (LTL) based on Event Calculus (EC) to generate persistent (i.e., constant) macro-actions, which guide Monte Carlo Tree Search (MCTS)-based POMDP solvers over a time horizon, significantly reducing inference time while ensuring robust performance. Such macro-actions are learnt via Inductive Logic Programming (ILP) from a few traces of execution (belief-action pairs), thus eliminating the need for manually designed heuristics and requiring only the specification of the POMDP transition model. In the Pocman and Rocksample benchmark scenarios, our learned macro-actions demonstrate increased expressiveness and generality when compared to time-independent heuristics, indeed offering substantial computational efficiency improvements.