# Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation

arXiv: https://arxiv.org/abs/2210.15751 · HTML: https://mlsanity.com/paper/2210.15751/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2022.

## Built on by (most cited first, 10 of 10)

- [EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generation](https://mlsanity.com/paper/2412.18907/) (2024)
- [Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning](https://mlsanity.com/paper/2602.12405/) (2026)
- [One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs](https://mlsanity.com/paper/2408.12674/) (2024)
- [Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulation](https://mlsanity.com/paper/2606.14188/) (2026)
- [TopoCut: Learning Multi-Step Cutting with Spectral Rewards and Discrete Diffusion Policies](https://mlsanity.com/paper/2509.19712/) (2025)
- [Learning Multimodal Attention for Manipulating Deformable Objects with Changing States](https://mlsanity.com/paper/2309.14837/) (2023)
- [Implicit Contact Diffuser: Sequential Contact Reasoning with Latent Point Cloud Diffusion](https://mlsanity.com/paper/2410.16571/) (2024)
- [Deformable Cluster Manipulation via Whole-Arm Policy Learning](https://mlsanity.com/paper/2507.17085/) (2025)
- [Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RL](https://mlsanity.com/paper/2605.26371/) (2026)
- [BridgeMatch: Conditional Transport Bridges in Matching Matrix Space for 3D Deformable Registration](https://mlsanity.com/paper/2609.11472/) (2026)

## Abstract (arXiv)

Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels. Previous methods typically either focus on short-horizon tasks or make strong assumptions that full-state information is available, which prevents their use on deformable objects. In this paper, we propose PlAnning with Spatial-Temporal Abstraction (PASTA), which incorporates both spatial abstraction (reasoning about objects and their relations to each other) and temporal abstraction (reasoning over skills instead of low-level actions). Our framework maps high-dimension 3D observations such as point clouds into a set of latent vectors and plans over skill sequences on top of the latent set representation. We show that our method can effectively perform challenging sequential deformable object manipulation tasks in the real world, which require combining multiple tool-use skills such as cutting with a knife, pushing with a pusher, and spreading the dough with a roller.