# TiPToP: A Modular Open-Vocabulary Robot Manipulation System That Plans

arXiv: https://arxiv.org/abs/2603.09971 · HTML: https://mlsanity.com/paper/2603.09971/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2026.

## Built on by (most cited first, 8 of 8)

- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [Addressing the Orchestration Gap in Generalist Robots via Physical Agency](https://mlsanity.com/paper/2607.21725/) (2026)
- [Improving Robotic Generalist Policies via Flow Reversal Steering](https://mlsanity.com/paper/2606.13675/) (2026)
- [DATAFARM: Distribution-Aligned Task and Motion Planning for Fine-Tuning Vision-Language-Action Models](https://mlsanity.com/paper/2609.12316/) (2026)
- [DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation](https://mlsanity.com/paper/2609.31112/) (2026)
- [Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planning](https://mlsanity.com/paper/2605.15975/) (2026)
- [TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuning](https://mlsanity.com/paper/2609.28314/) (2026)
- [LADDER: A Long-Horizon Robot Manipulation Benchmark](https://mlsanity.com/paper/2610.18053/) (2026)

## Abstract (arXiv)

We present TiPToP, a modular manipulation system that integrates pretrained foundation models with a GPU-accelerated Task and Motion Planner to solve tasks directly from RGB images and natural language. TiPToP composes perception, planning, and execution modules and requires no robot training data. It can be deployed on a standard DROID setup in under an hour and adapted to new embodiments with minimal effort. We evaluate TiPToP against π0.5-DROID, a state-of-the-art VLA fine-tuned on 350 hours of demonstrations, across two real-world DROID setups (one operated by an external team) and simulation, where TiPToP attains a higher average success rate and faster average completion time. We also evaluate on the MolmoSpaces benchmark, where TiPToP ranks first overall on pick and pick-and-place tasks among methods not trained on in-distribution data. We further show that TiPToP's modularity enables us to trace failures to specific components, revealing where to target improvements. We release TiPToP open-source to serve as a reproducible baseline and to enable further research on modular manipulation systems. Project website and code: https://tiptop-robot.github.io