# Optimistic Reinforcement Learning-Based Skill Insertions for Task and Motion Planning

arXiv: https://arxiv.org/abs/2510.14065 · HTML: https://mlsanity.com/paper/2510.14065/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 18 of 18)

- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning](https://mlsanity.com/paper/1802.08705/) (2018)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [Learning Symbolic Operators for Task and Motion Planning](https://mlsanity.com/paper/2103.00589/) (2021)
- [Learning Neuro-Symbolic Skills for Bilevel Planning](https://mlsanity.com/paper/2206.10680/) (2022)
- [PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making](https://mlsanity.com/paper/1804.07779/) (2018)
- [Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image](https://mlsanity.com/paper/2006.05398/) (2020)
- [Learning compositional models of robot skills for task and motion planning](https://mlsanity.com/paper/2006.06444/) (2020)
- [Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances](https://mlsanity.com/paper/2108.04145/) (2021)
- [Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/1904.06703/) (2019)
- [Object-Centric Task and Motion Planning in Dynamic Environments](https://mlsanity.com/paper/1911.04679/) (2019)
- [Discovering State and Action Abstractions for Generalized Task and Motion Planning](https://mlsanity.com/paper/2109.11082/) (2021)
- [Learning STRIPS Action Models with Classical Planning](https://mlsanity.com/paper/1903.01153/) (2019)
- [Efficient learning of goal-oriented push-grasping synergy in clutter](https://mlsanity.com/paper/2103.05405/) (2021)
- [Learning to Search in Task and Motion Planning with Streams](https://mlsanity.com/paper/2111.13144/) (2021)
- [Learning Reusable Manipulation Strategies](https://mlsanity.com/paper/2311.03293/) (2023)
- [Sampling-Based Methods for Factored Task and Motion Planning](https://mlsanity.com/paper/1801.00680/) (2018)
- [Backward-Forward Search for Manipulation Planning](https://mlsanity.com/paper/1604.03468/) (2016)

## Abstract (arXiv)

Task and motion planning (TAMP) for robotics manipulation necessitates long-horizon reasoning involving versatile actions and skills. While deterministic actions can be crafted by sampling or optimizing with certain constraints, planning actions with uncertainty, i.e., probabilistic actions, remains a challenge for TAMP. On the contrary, Reinforcement Learning (RL) excels in acquiring versatile, yet short-horizon, manipulation skills that are robust with uncertainties. In this letter, we design a method that integrates RL skills into TAMP pipelines. Besides the policy, a RL skill is defined with data-driven logical components that enable the skill to be deployed by symbolic planning. A plan refinement sub-routine is designed to further tackle the inevitable effect uncertainties. In the experiments, we compare our method with baseline hierarchical planning from both TAMP and RL fields and illustrate the strength of the method. The results show that by embedding RL skills, we extend the capability of TAMP to domains with probabilistic skills, and improve the planning efficiency compared to the previous methods.