# Learning Logic Specifications for Soft Policy Guidance in POMCP

arXiv: https://arxiv.org/abs/2303.09172 · HTML: https://mlsanity.com/paper/2303.09172/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 16 Mar 2023.

## Builds on (most cited first, 13 of 13)

- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [ASP-Core-2 Input Language Format](https://mlsanity.com/paper/1911.04326/) (2019)
- [Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications](https://mlsanity.com/paper/1807.06333/) (2018)
- [Learning First-Order Symbolic Representations for Planning from the Structure of the State Space](https://mlsanity.com/paper/1909.05546/) (2019)
- [Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes](https://mlsanity.com/paper/1302.1525/) (2013)
- [Learning First-Order Representations for Planning from Black-Box States: New Results](https://mlsanity.com/paper/2105.10830/) (2021)
- [Autonomous task planning and situation awareness in robotic surgery](https://mlsanity.com/paper/2004.08911/) (2020)
- [Logic programming for deliberative robotic task planning](https://mlsanity.com/paper/2301.07550/) (2023)
- [Deliberation in autonomous robotic surgery: a framework for handling anatomical uncertainty](https://mlsanity.com/paper/2203.05438/) (2022)
- [POMP: Pomcp-based Online Motion Planning for active visual search in indoor environments](https://mlsanity.com/paper/2009.08140/) (2020)
- [POMP++: Pomcp-based Active Visual Search in unknown indoor environments](https://mlsanity.com/paper/2107.00914/) (2021)
- [Identification of Unexpected Decisions in Partially Observable Monte-Carlo Planning: a Rule-Based Approach](https://mlsanity.com/paper/2012.12732/) (2020)
- [Rule-based Shielding for Partially Observable Monte-Carlo Planning](https://mlsanity.com/paper/2104.13791/) (2021)

## Built on by (most cited first, 7 of 7)

- [BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximations](https://mlsanity.com/paper/2306.00249/) (2023)
- [Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach](https://mlsanity.com/paper/2402.19265/) (2024)
- [Online inductive learning from answer sets for efficient reinforcement learning exploration](https://mlsanity.com/paper/2501.07445/) (2025)
- [Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback](https://mlsanity.com/paper/2501.07507/) (2025)
- [Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time](https://mlsanity.com/paper/2505.03668/) (2025)
- [Belief-State Query Policies for User-Aligned POMDPs](https://mlsanity.com/paper/2405.15907/) (2024)
- [Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processes](https://mlsanity.com/paper/2607.12924/) (2026)

## Abstract (arXiv)

Partially Observable Monte Carlo Planning (POMCP) is an efficient solver for Partially Observable Markov Decision Processes (POMDPs). It allows scaling to large state spaces by computing an approximation of the optimal policy locally and online, using a Monte Carlo Tree Search based strategy. However, POMCP suffers from sparse reward function, namely, rewards achieved only when the final goal is reached, particularly in environments with large state spaces and long horizons. Recently, logic specifications have been integrated into POMCP to guide exploration and to satisfy safety requirements. However, such policy-related rules require manual definition by domain experts, especially in real-world scenarios. In this paper, we use inductive logic programming to learn logic specifications from traces of POMCP executions, i.e., sets of belief-action pairs generated by the planner. Specifically, we learn rules expressed in the paradigm of answer set programming. We then integrate them inside POMCP to provide soft policy bias toward promising actions. In the context of two benchmark scenarios, rocksample and battery, we show that the integration of learned rules from small task instances can improve performance with fewer Monte Carlo simulations and in larger task instances. We make our modified version of POMCP publicly available at https://github.com/GiuMaz/pomcp_clingo.git.