mlsanityData to 14 Oct 2026

NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos

arXiv 2510.08568 ↗First posted 9 Oct 2025cs.RO cs.AI cs.CVcited by 40

Not on the map. Why?

Abstract (arXiv)

Enabling robots to execute novel manipulation tasks zero-shot is a central goal in robotics. Most existing methods assume in-distribution tasks or rely on fine-tuning with embodiment-matched data, limiting transfer across platforms. We present NovaFlow, an autonomous manipulation framework that converts a task description into an actionable plan for a target robot without any demonstrations. Given a task description, NovaFlow synthesizes a video using a video generation model and distills it into 3D actionable object flow using off-the-shelf perception modules. From the object flow, it computes relative poses for rigid objects and realizes them as robot actions via grasp proposals and trajectory optimization. For deformable objects, this flow serves as a tracking objective for model-based planning with a particle-based dynamics model. By decoupling task understanding from low-level control, NovaFlow naturally transfers across embodiments. We validate on rigid, articulated, and deformable object manipulation tasks using a table-top Franka arm and a Spot quadrupedal mobile robot, and achieve effective zero-shot execution without demonstrations or embodiment-specific training. Project website: https://novaflow.lhy.xyz/.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 60 of its 65 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

60 of 65 references.

References · 60

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  11. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  12. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
38 more
  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  2. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  4. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  5. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  7. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  8. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  9. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  10. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  11. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
  12. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
  13. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  14. 2020Consistent Video Depth Estimationcited by 220
  15. 2025Video models are zero-shot learners and reasonerscited by 220
  16. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  17. 2025Unified Video Action Modelcited by 205
  18. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
  19. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  20. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 156
  21. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 153
  22. 2024Flow as the Cross-Domain Manipulation Interfacecited by 132
  23. 2025A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulationcited by 129
  24. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 124
  25. 2023Compositional Foundation Models for Hierarchical Planningcited by 95
  26. 2025Video Generators are Robot Policiescited by 92
  27. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  28. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 87
  29. 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 87
  30. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 85
  31. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 69
  32. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 54
  33. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 52
  34. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  35. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 38
  36. 20253DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Modelcited by 37
  37. 2023FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projectioncited by 34
  38. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 32

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 36.

40 papers.

Sort:

By area:

  1. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  2. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  3. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  4. 2026Interactive World Simulator for Robot Policy Training and Evaluationcited by 31
  5. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 30
  6. 2025ViPRA: Video Prediction for Robot Actionscited by 26
  7. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 26
  8. 2026VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generationcited by 23
  9. 2026EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Datacited by 20
  10. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 16
  11. 2025TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videoscited by 15
  12. 2026SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulationcited by 15
28 more
  1. 2026Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulationcited by 14
  2. 2026World Action Models: A Surveycited by 12
  3. 2026Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstructioncited by 11
  4. 2026LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Priorcited by 11
  5. 2026Masked Visual Actions for Unified World Modelingcited by 9
  6. 2026TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motioncited by 8
  7. 2026NovaPlan: Zero-Shot Long-Horizon Manipulation via Closed-Loop Video Language Planningcited by 8
  8. 2026Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulationcited by 8
  9. 2026Veo-Act: Enhancing VLA Policies with Frontier Video Modelscited by 5
  10. 2026Robot Learning from Human Videos: A Surveycited by 5
  11. 2026LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisitioncited by 5
  12. 2026μ0: A Scalable 3D Interaction-Trace World Modelcited by 3
  13. 2026Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratiocited by 3
  14. 2026Universal Manipulation Exoskeleton: Learning Compliant Whole-body Policies with Real-time Torque Feedbackcited by 2
  15. 2026Latent Goal Prediction from Language for Model-Based Planningcited by 2
  16. 2026GenVid2Robot: From Video Generation to Robot Manipulation via Rigid-Geometric Consistencycited by 2
  17. 2025In-Video Instructions: Visual Signals as Generative Controlcited by 1
  18. 2026AnchorD: Metric Grounding of Monocular Depth Using Factor Graphscited by 1
  19. 2026MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Predictioncited by 1
  20. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 1
  21. 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionablecited by 1
  22. 2026Human2Any: Human-to-Robot Transfer via Constraint-Aware Compositional Planningcited by 0
  23. 2026Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Modelscited by 0
  24. 2026DreamTraj: Generating 6-DoF Object Trajectories by Reading Unrendered Video Diffusion Latentscited by 0
  25. 2026Track, Articulate, Act: Generating Articulation from Casual Human Videoscited by 0
  26. 2026Dreaming the Sound of Contact: Leveraging Video and Audio Generation for Zero-Shot Force-Aware Manipulation and Data Generationcited by 0
  27. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventioncited by 0
  28. 2026CASTER: Formalizing Zero-Shot Manipulation from Synthetic Videos as Constrained Optimizationcited by 0