mlsanityData to 7 Oct 2026

Unified Video Action Model

arXiv 2503.00200 ↗First posted 28 Feb 2025cs.RO cs.CVcited by 195

Not on the map. Why?

Cited by 99 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A unified video and action model holds significant promise for robotics, where videos provide rich scene information for action prediction, and actions provide dynamics information for video prediction. However, effectively combining video generation and action prediction remains challenging, and current video generation-based methods struggle to match the performance of direct policy learning in action accuracy and inference speed. To bridge this gap, we introduce the Unified Video Action model (UVA), which jointly optimizes video and action predictions to achieve both high accuracy and efficient action inference. The key lies in learning a joint video-action latent representation and decoupling video-action decoding. The joint latent representation bridges the visual and action domains, effectively modeling the relationship between video and action sequences. Meanwhile, the decoupled decoding, powered by two lightweight diffusion heads, enables high-speed action inference by bypassing video generation during inference. Such a unified framework further enables versatile functionality through masked input training. By selectively masking actions or videos, a single model can tackle diverse tasks beyond policy learning, such as forward and inverse dynamics modeling and video generation. Via an extensive set of experiments, we demonstrate that UVA can serve as a general-purpose solution for a wide range of robotics tasks, such as policy learning, forward/inverse dynamics and video observation prediction, without compromising performance compared to methods tailored for specific applications. Results are best viewed on https://unified-video-action-model.github.io/.

Key facts. Cited by 195 papers in the map (data through 7 Oct 2026). Builds on 49 of its 50 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

49 of 50 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  12. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
37 more
  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2023Consistency Modelscited by 1,778
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  8. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  10. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  11. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  12. 2024Genie: Generative Interactive Environmentscited by 706
  13. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  14. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  15. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  16. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  17. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  18. 2021Implicit Behavioral Cloningcited by 442
  19. 2023Improved Techniques for Training Consistency Modelscited by 430
  20. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  21. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  22. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  23. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  24. 2019Scaling Autoregressive Video Modelscited by 189
  25. 2024Autoregressive Video Generation without Vector Quantizationcited by 180
  26. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  27. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  28. 2024Flow as the Cross-Domain Manipulation Interfacecited by 126
  29. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
  30. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  31. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
  32. 2024Prediction with Action: Visual Policy Learning via Joint Denoising Processcited by 64
  33. 2023Robot Learning with Sensorimotor Pre-trainingcited by 51
  34. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 44
  35. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 40
  36. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Modelscited by 33
  37. 2025Action-Free Reasoning for Policy Generalizationcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 55; 2026: 140.

195 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2026World Action Models are Zero-shot Policiescited by 411
  3. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  4. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  5. 2026Causal World Modeling for Robot Controlcited by 289
  6. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  7. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  8. 2025Motus: A Unified Latent Action World Modelcited by 231
  9. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  10. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  11. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  12. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
38 more
  1. 2025Unified Vision-Language-Action Modelcited by 128
  2. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  3. 2025FLARE: Robot Learning with Implicit World Modelingcited by 105
  4. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 83
  5. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  6. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  7. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  8. 2025Large Video Planner Enables Generalizable Robot Controlcited by 67
  9. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 61
  10. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
  11. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  12. 2026World Model for Robot Learning: A Comprehensive Surveycited by 42
  13. 2025DynaGuide: Steering Diffusion Polices with Active Dynamic Guidancecited by 39
  14. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 38
  15. 2025NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videoscited by 38
  16. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
  17. 2025Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Processcited by 37
  18. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generationcited by 36
  19. 2026Do World Action Models Generalize Better than VLAs? A Robustness Studycited by 35
  20. 2026Hierarchical Planning with Latent World Modelscited by 34
  21. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 33
  22. 2025HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Modelscited by 30
  23. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
  24. 2026Native Video-Action Pretraining for Generalizable Robot Controlcited by 30
  25. 2026Interactive World Simulator for Robot Policy Training and Evaluationcited by 27
  26. 2026Self-Supervised Flow Matching for Scalable Multi-Modal Synthesiscited by 25
  27. 2025ViPRA: Video Prediction for Robot Actionscited by 24
  28. 2026Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectoriescited by 24
  29. 2025A Survey of Interactive Generative Videocited by 23
  30. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  31. 2025MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulationcited by 22
  32. 2025Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flowcited by 21
  33. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
  34. 2026Light-WAM: Efficient World Action Models with State-Fusion Action Decodingcited by 19
  35. 2025AMPLIFY: Actionless Motion Priors for Robot Learning from Videoscited by 18
  36. 2025Robot Learning from a Physical World Modelcited by 18
  37. 2026When to Trust Imagination: Adaptive Action Execution for World Action Modelscited by 18
  38. 2026LIBERO-X: Robustness Litmus for Vision-Language-Action Modelscited by 17

50 most cited of 195 All 195 (JSON)