mlsanityData to 7 Oct 2026

Robots Need More than VLA and World Models

arXiv 2606.06556 ↗First posted 4 Jun 2026cs.ROcited by 2

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

Generalist robot intelligence is often framed as a policy-scaling problem: collect more robot demonstrations, train larger Vision-Language-Action (VLA) models, and expect broader generalisation. In this position paper, we argue that this framing is incomplete. The central bottleneck is not only policy learning, but the absence of mechanisms that convert the world's abundant unstructured behavioural data into grounded robot supervision. Human motion, internet video, simulation rollouts, and interactive demonstrations contain rich information about tasks, goals, contacts, failures, and physical constraints, yet most of this information is not directly usable by robot policies because it lacks embodiment-specific action labels, task semantics, and reward structure. We identify four missing components for the next generation of robotics: data interfaces for autolabelling unstructured behaviour, embodiment interfaces for retargeting human motion to robot actions, world-model interfaces for physics-grounded 3D reasoning, and reward interfaces for inferring task progress and success from video and language. We survey recent progress in robot foundation models, cross-embodiment datasets, learning from video, world models, and reward modelling, and propose a research agenda for building robotics systems that can learn not only from robot demonstrations, but from the broader physical world.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 44 of its 114 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

44 of 114 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026World Action Models are Zero-shot Policies17 Feb
  2. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation7 Jan
  3. 2025World Models That Know When They Don't Know - Controllable Video Generation with Calibrated Uncertainty5 Dec
  4. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulation25 Aug
  5. 2025ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation29 Jun
  6. 2025Object-Centric World Model for Language-Guided Manipulation8 Mar
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  8. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb

Other references · 36

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  5. 2018World Modelscited by 1,444
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  8. 2020Mastering Atari with Discrete World Modelscited by 990
  9. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  11. 2024Genie: Generative Interactive Environmentscited by 706
  12. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
24 more
  1. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  2. 2023Learning Interactive Real-World Simulatorscited by 440
  3. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  4. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  5. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  6. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  7. 2024Latent Action Pretraining from Videoscited by 302
  8. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
  9. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  10. 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
  11. 2025LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instructioncited by 40
  12. 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Controlcited by 32
  13. 2025FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenizationcited by 21
  14. 2025GraphCoT-VLA: A 3D Spatial-Aware Reasoning Vision-Language-Action Model for Robotic Manipulation with Ambiguous Instructionscited by 19
  15. 2026ActionCodec: What Makes for Good Action Tokenizerscited by 17
  16. 2025Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robotscited by 15
  17. 2025GRaD-Nav++: Vision-Language Model Enabled Visual Drone Navigation with Gaussian Radiance Fields and Differentiable Dynamicscited by 14
  18. 2025Spatial Traces: Enhancing VLA Models with Spatial-Temporal Understandingcited by 11
  19. 2026Foundational World Models Accurately Detect Bimanual Manipulator Failurescited by 11
  20. 2025Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videoscited by 10
  21. 2026HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulationcited by 10
  22. 2026ContactGaussian-WM: Learning Physics-Grounded World Model from Videoscited by 7
  23. 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Datacited by 3
  24. 2025Avi: Action from Volumetric Inferencecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Action-Free Reasoning for Policy Generalization
  2. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Models
  3. 2026Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence
  4. 2025UniJEPA: Enhancing Robot Policy via Unified Continuous and Discrete Representation Learning
  5. 2026Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data
  6. 2026Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models
  7. 2024Octo: An Open-Source Generalist Robot Policy
  8. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Model