# Task and Skill Planning: Hierarchical Robot Planning with Black-Box Skills

arXiv: https://arxiv.org/abs/2504.17901 · HTML: https://mlsanity.com/paper/2504.17901/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Builds on (most cited first, 18 of 18)

- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study](https://mlsanity.com/paper/1404.3785/) (2014)
- [CNOS: A Strong Baseline for CAD-based Novel Object Segmentation](https://mlsanity.com/paper/2307.11067/) (2023)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [Learning Neuro-Symbolic Skills for Bilevel Planning](https://mlsanity.com/paper/2206.10680/) (2022)
- [Learning compositional models of robot skills for task and motion planning](https://mlsanity.com/paper/2006.06444/) (2020)
- [LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation](https://mlsanity.com/paper/2210.12631/) (2022)
- [Entity-Centric Reinforcement Learning for Object Manipulation from Pixels](https://mlsanity.com/paper/2404.01220/) (2024)
- [Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation](https://mlsanity.com/paper/2109.08771/) (2021)
- [STAP: Sequencing Task-Agnostic Policies](https://mlsanity.com/paper/2210.12250/) (2022)
- [Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness](https://mlsanity.com/paper/2403.10454/) (2024)
- [Anytime Integrated Task and Motion Policies for Stochastic Environments](https://mlsanity.com/paper/1904.13006/) (2019)
- [Task and Motion Planning for Execution in the Real](https://mlsanity.com/paper/2406.03641/) (2024)
- [Planning for Multi-stage Forceful Manipulation](https://mlsanity.com/paper/2101.02679/) (2021)
- [Learning Preconditions of Hybrid Force-Velocity Controllers for Contact-Rich Manipulation](https://mlsanity.com/paper/2206.12728/) (2022)

## Abstract (arXiv)

Task and motion planning (TAMP) is a well-established approach for solving long-horizon robot planning problems. Although TAMP methods have historically assumed that each task-level robot action, or skill, can be reduced to kinematic motion planning, recent work has explored integrating closed-loop controllers and learned skills into TAMP-style systems. Our approach integrates pre-existing, heterogeneous robot skills--including learned, force-controlled, and black-box policies--into a hierarchical planner while preserving the object-centric failure reasoning of typical TAMP solvers. We leverage Composable Interaction Primitives (CIPs) to synthesize head and tail motion plans bridging consecutive skills, facilitating both planning-time refinement and execution-time adjustment. We validate our Task and Skill Planning (TASP) approach through real-world experiments on a bimanual manipulator and a mobile manipulator, demonstrating that CIPs enable diverse robots to combine heterogeneous skills to solve complex, long-horizon tasks, including multi-room mobile manipulation problems with non-monotonic task structure.