# Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics

arXiv: https://arxiv.org/abs/2408.14769 · HTML: https://mlsanity.com/paper/2408.14769/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2024.

## Built on by (most cited first, 4 of 4)

- [Mobi-π: Mobilizing Your Robot Learning Policy](https://mlsanity.com/paper/2505.23692/) (2025)
- [Structured 4D Latent Predictive Model for Robot Planning](https://mlsanity.com/paper/2607.01166/) (2026)
- [Hierarchical DLO Routing with Reinforcement Learning and In-Context Vision-language Models](https://mlsanity.com/paper/2510.19268/) (2025)
- [RoboVista: Evaluating Vision Language Models for Diverse Robot Applications](https://mlsanity.com/paper/2607.04610/) (2026)

## Abstract (arXiv)

We present Points2Plans, a framework for composable planning with a relational dynamics model that enables robots to solve long-horizon manipulation tasks from partial-view point clouds. Given a language instruction and a point cloud of the scene, our framework initiates a hierarchical planning procedure, whereby a language model generates a high-level plan and a sampling-based planner produces constraint-satisfying continuous parameters for manipulation primitives sequenced according to the high-level plan. Key to our approach is the use of a relational dynamics model as a unifying interface between the continuous and symbolic representations of states and actions, thus facilitating language-driven planning from high-dimensional perceptual input such as point clouds. Whereas previous relational dynamics models require training on datasets of multi-step manipulation scenarios that align with the intended test scenarios, Points2Plans uses only single-step simulated training data while generalizing zero-shot to a variable number of steps during real-world evaluations. We evaluate our approach on tasks involving geometric reasoning, multi-object interactions, and occluded object reasoning in both simulated and real-world settings. Results demonstrate that Points2Plans offers strong generalization to unseen long-horizon tasks in the real world, where it solves over 85% of evaluated tasks while the next best baseline solves only 50%.