# TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion arXiv: https://arxiv.org/abs/2601.18323 · HTML: https://mlsanity.com/paper/2601.18323/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Embodied and robot world models, one of 11 directions in World models. - Builds on 28 of its 43 references that are in the map. - First posted on arXiv 26 Jan 2026. ## Placed in - World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/) - Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/) - Vision-language-action models / [World model-based VLA](https://mlsanity.com/vla/world-model-based-vla/) ## Builds on (most cited first, 28 of 28) - [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024) - [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024) - [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025) - [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022) - [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025) - [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025) - [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023) - [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023) - [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024) - [Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations](https://mlsanity.com/paper/2412.14803/) (2024) - [DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge](https://mlsanity.com/paper/2507.04447/) (2025) - [MolmoAct: Action Reasoning Models that can Reason in Space](https://mlsanity.com/paper/2508.07917/) (2025) - [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024) - [MiniMax-01: Scaling Foundation Models with Lightning Attention](https://mlsanity.com/paper/2501.08313/) (2025) - [Learning to Act from Actionless Videos through Dense Correspondences](https://mlsanity.com/paper/2310.08576/) (2023) - [GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data](https://mlsanity.com/paper/2505.03233/) (2025) - [DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping](https://mlsanity.com/paper/2502.20900/) (2025) - [SpatialTrackerV2: 3D Point Tracking Made Easy](https://mlsanity.com/paper/2507.12462/) (2025) - [WoW: Towards a World omniscient World model Through Embodied Interaction](https://mlsanity.com/paper/2509.22642/) (2025) - [Learning Robotic Manipulation through Visual Planning and Acting](https://mlsanity.com/paper/1905.04411/) (2019) - [NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos](https://mlsanity.com/paper/2510.08568/) (2025) - [EVA: An Embodied World Model for Future Video Anticipation](https://mlsanity.com/paper/2410.15461/) (2024) - [AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation](https://mlsanity.com/paper/2507.12768/) (2025) - [MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis](https://mlsanity.com/paper/2506.18897/) (2025) - [Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modeling](https://mlsanity.com/paper/2512.03044/) (2025) - [Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robots](https://mlsanity.com/paper/2509.02530/) (2025) ## Built on by (most cited first, 8 of 8) - [World Model for Robot Learning: A Comprehensive Survey](https://mlsanity.com/paper/2605.00080/) (2026) - [ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?](https://mlsanity.com/paper/2606.19531/) (2026) - [Mask World Model: Predicting What Matters for Robust Robot Policy Learning](https://mlsanity.com/paper/2604.19683/) (2026) - [World Action Models: A Survey](https://mlsanity.com/paper/2606.20781/) (2026) - [HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models](https://mlsanity.com/paper/2605.10942/) (2026) - [Veo-Act: Enhancing VLA Policies with Frontier Video Models](https://mlsanity.com/paper/2604.04502/) (2026) - [TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-Training](https://mlsanity.com/paper/2607.02840/) (2026) - [World Models for Embodied Intelligence: From Plausible to Controllable to Actionable](https://mlsanity.com/paper/2609.16697/) (2026) ## Abstract (arXiv) The vision-language-action (VLA) paradigm has enabled powerful robotic control by leveraging vision-language models, but its reliance on large-scale, high-quality robot data limits its generalization. Generative world models offer a promising alternative for general-purpose embodied AI, yet a critical gap remains between their pixel-level plans and physically executable actions. To this end, we propose the Tool-Centric Inverse Dynamics Model (TC-IDM). By focusing on the tool's imagined trajectory as synthesized by the world model, TC-IDM establishes a robust intermediate representation that bridges the gap between visual planning and physical control. TC-IDM extracts the tool's point cloud trajectories via segmentation and 3D motion estimation from generated videos. Considering diverse tool attributes, our architecture employs decoupled action heads to project these planned trajectories into 6-DoF end-effector motions and corresponding control signals. This plan-and-translate paradigm not only supports a wide range of end-effectors but also significantly improves viewpoint invariance. Furthermore, it exhibits strong generalization capabilities across long-horizon and out-of-distribution tasks, including interacting with deformable objects. In real-world evaluations, the world model with TC-IDM achieves an average success rate of 61.11 percent, with 77.7 percent on simple tasks and 38.46 percent on zero-shot deformable object tasks. It substantially outperforms end-to-end VLA-style baselines and other inverse dynamics models.