mlsanityData to 7 Oct 2026

UniVLA: Learning to Act Anywhere with Task-centric Latent Actions

arXiv 2505.06111 ↗First posted 9 May 2025cs.RO cs.AI cs.LGcited by 386

On the map: Vision-language-action models → Human-video and latent-action pretraining

Abstract (arXiv)

A generalist robot should perform effectively across various environments. However, most existing approaches heavily rely on scaling action-annotated data to enhance their capabilities. Consequently, they are often limited to single physical specification and struggle to learn transferable knowledge across different embodiments and environments. To confront these limitations, we propose UniVLA, a new framework for learning cross-embodiment vision-language-action (VLA) policies. Our key innovation is to derive task-centric action representations from videos with a latent action model. This enables us to exploit extensive data across a wide spectrum of embodiments and perspectives. To mitigate the effect of task-irrelevant dynamics, we incorporate language instructions and establish a latent action model within the DINO feature space. Learned from internet-scale videos, the generalist policy can be deployed to various robots through efficient latent action decoding. We obtain state-of-the-art results across multiple manipulation and navigation benchmarks, as well as real-robot deployments. UniVLA achieves superior performance over OpenVLA with less than 1/20 of pretraining compute and 1/10 of downstream data. Continuous performance improvements are observed as heterogeneous data, even including human videos, are incorporated into the training pipeline. The results underscore UniVLA's potential to facilitate scalable and efficient robot policy learning.

Key facts. Cited by 386 papers in the map (data through 7 Oct 2026). Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. Builds on 87 of its 96 references that are in the map. First posted on arXiv 9 May 2025.

Builds on

87 of 96 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Latent Action Pretraining from Videos15 Oct

Other references · 86

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  10. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  11. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  12. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
38 more
  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  4. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  5. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  8. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  9. 2016Deep Variational Information Bottleneckcited by 915
  10. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  11. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  13. 2024Genie: Generative Interactive Environmentscited by 706
  14. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  15. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  16. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  17. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  18. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  19. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  20. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  21. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  22. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  23. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  24. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  25. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  26. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  27. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  28. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  29. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  30. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  31. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  32. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  33. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
  34. 2022GNM: A General Navigation Model to Drive Any Robotcited by 161
  35. 2023Video Language Planningcited by 160
  36. 2024Behavior Generation with Latent Actionscited by 159
  37. 2023Structured World Models from Human Videoscited by 144
  38. 2016Variational Autoencoder for Deep Learning of Images, Labels and Captionscited by 137

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 84; 2026: 302.

386 papers.

Sort:

By area:

  1. 2026World Action Models are Zero-shot Policiescited by 411
  2. 2026Causal World Modeling for Robot Controlcited by 289
  3. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  4. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  5. 2025Motus: A Unified Latent Action World Modelcited by 231
  6. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  7. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
  8. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  9. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  10. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  11. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
  12. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
38 more
  1. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  2. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  3. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  4. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
  5. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  6. 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 85
  7. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 83
  8. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  9. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  10. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 70
  11. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  12. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  13. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
  14. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  15. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  16. 2025villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Modelscited by 58
  17. 2025World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Trainingcited by 57
  18. 2026Motubrain: An Advanced World Action Model for Robot Controlcited by 54
  19. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  20. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 49
  21. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 47
  22. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  23. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 46
  24. 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policiescited by 46
  25. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 43
  26. 2025BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulationcited by 42
  27. 2026Learning Latent Action World Models In The Wildcited by 42
  28. 2026Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Executioncited by 42
  29. 2026World Model for Robot Learning: A Comprehensive Surveycited by 42
  30. 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routingcited by 41
  31. 2025SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruningcited by 39
  32. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  33. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 37
  34. 2026Do World Action Models Generalize Better than VLAs? A Robustness Studycited by 35
  35. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 34
  36. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34
  37. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  38. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 33

50 most cited of 386 All 386 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Unified Vision-Language-Action Model
  2. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
  3. 2026Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
  4. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control
  5. 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action
  6. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
  7. 2026GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture
  8. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model