# SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning

arXiv: https://arxiv.org/abs/2012.13037 · HTML: https://mlsanity.com/paper/2012.13037/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2020.

## Built on by (most cited first, 23 of 23)

- [Learning Symbolic Operators for Task and Motion Planning](https://mlsanity.com/paper/2103.00589/) (2021)
- [Learning Neuro-Symbolic Skills for Bilevel Planning](https://mlsanity.com/paper/2206.10680/) (2022)
- [Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning](https://mlsanity.com/paper/2105.14074/) (2021)
- [LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation](https://mlsanity.com/paper/2210.12631/) (2022)
- [Neuro-Symbolic World Models for Adapting to Open World Novelty](https://mlsanity.com/paper/2301.06294/) (2023)
- [Verifiable and Compositional Reinforcement Learning Systems](https://mlsanity.com/paper/2106.05864/) (2021)
- [RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments](https://mlsanity.com/paper/2206.12493/) (2022)
- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework](https://mlsanity.com/paper/2204.10358/) (2022)
- [NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds](https://mlsanity.com/paper/2401.03546/) (2024)
- [Verifiable Reinforcement Learning Systems via Compositionality](https://mlsanity.com/paper/2309.06420/) (2023)
- [Curiosity-Driven Imagination: Discovering Plan Operators and Learning Associated Policies for Open-World Adaptation](https://mlsanity.com/paper/2503.04931/) (2025)
- [SLAP: Shortcut Learning for Abstract Planning](https://mlsanity.com/paper/2511.01107/) (2025)
- [Controller Synthesis for Omega-Regular and Steady-State Specifications](https://mlsanity.com/paper/2106.02951/) (2021)
- [Few-Shot Neuro-Symbolic Imitation Learning for Long-Horizon Planning and Acting](https://mlsanity.com/paper/2508.21501/) (2025)
- [Creativity of AI: Hierarchical Planning Model Learning for Facilitating Deep Reinforcement Learning](https://mlsanity.com/paper/2112.09836/) (2021)
- [Learning Symbolic Operators: A Neurosymbolic Solution for Autonomous Disassembly of Electric Vehicle Battery](https://mlsanity.com/paper/2206.03027/) (2022)
- [Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning](https://mlsanity.com/paper/2603.11351/) (2026)
- [Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments](https://mlsanity.com/paper/2302.14208/) (2023)
- [NSP: A Neuro-Symbolic Natural Language Navigational Planner](https://mlsanity.com/paper/2409.06859/) (2024)
- [Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics](https://mlsanity.com/paper/2601.16985/) (2026)
- [Build on Priors: Vision--Language--Guided Neuro-Symbolic Imitation Learning for Data-Efficient Real-World Robot Manipulation](https://mlsanity.com/paper/2604.03759/) (2026)
- [Macro-Operator Generation and Predicate Selection for TAMP Operator Learning](https://mlsanity.com/paper/2608.23629/) (2026)

## Abstract (arXiv)

Symbolic planning models allow decision-making agents to sequence actions in arbitrary ways to achieve a variety of goals in dynamic domains. However, they are typically handcrafted and tend to require precise formulations that are not robust to human error. Reinforcement learning (RL) approaches do not require such models, and instead learn domain dynamics by exploring the environment and collecting rewards. However, RL approaches tend to require millions of episodes of experience and often learn policies that are not easily transferable to other tasks. In this paper, we address one aspect of the open problem of integrating these approaches: how can decision-making agents resolve discrepancies in their symbolic planning models while attempting to accomplish goals? We propose an integrated framework named SPOTTER that uses RL to augment and support ("spot") a planning agent by discovering new operators needed by the agent to accomplish goals that are initially unreachable for the agent. SPOTTER outperforms pure-RL approaches while also discovering transferable symbolic knowledge and does not require supervision, successful plan traces or any a priori knowledge about the missing planning operator.