mlsanityData to 7 Oct 2026

ST4VLA: Spatially Guided Training for Vision-Language-Action Models

arXiv 2602.10109 ↗First posted 10 Feb 2026cs.ROcited by 8

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Large vision-language models (VLMs) excel at multimodal understanding but fall short when extended to embodied tasks, where instructions must be transformed into low-level motor actions. We introduce ST4VLA, a dual-system Vision-Language-Action framework that leverages Spatial Guided Training to align action learning with spatial priors in VLMs. ST4VLA includes two stages: (i) spatial grounding pre-training, which equips the VLM with transferable priors via scalable point, box, and trajectory prediction from both web-scale and robot-specific data, and (ii) spatially guided action post-training, which encourages the model to produce richer spatial priors to guide action generation via spatial prompting. This design preserves spatial grounding during policy learning and promotes consistent optimization across spatial and action objectives. Empirically, ST4VLA achieves substantial improvements over vanilla VLA, with performance increasing from 66.1 -> 84.6 on Google Robot and from 54.7 -> 73.2 on WidowX Robot, establishing new state-of-the-art results on SimplerEnv. It also demonstrates stronger generalization to unseen objects and paraphrased instructions, as well as robustness to long-horizon perturbations in real-world settings. These results highlight scalable spatially guided training as a promising direction for robust, generalizable robot learning. Source code, data and models are released at https://internrobotics.github.io/internvla-m1.github.io/

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 84 of its 91 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

84 of 91 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025RoboBrain 2.0 Technical Report2 Jul
  2. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics4 Jun
  3. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics2 Jun
  4. 2025Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces30 May
  5. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better29 May
  6. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models27 Mar
  7. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
  8. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
  9. 2024LLaRA: Supercharging Robot Learning Data for Vision-Language Policy28 Jun
  10. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning17 Jun
  11. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics15 Jun
  12. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 72

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
38 more
  1. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  2. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  3. 2016Modeling Context in Referring Expressionscited by 1,054
  4. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  6. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  9. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  10. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  11. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  12. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  13. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  14. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  15. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  16. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  17. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  18. 2024Latent Action Pretraining from Videoscited by 302
  19. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  20. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  21. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  22. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
  23. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
  24. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  25. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 205
  26. 2024RT-H: Action Hierarchies Using Languagecited by 204
  27. 2025Unified Video Action Modelcited by 195
  28. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  29. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  30. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
  31. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  32. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
  33. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  34. 2025Unified Vision-Language-Action Modelcited by 128
  35. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  36. 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 112
  37. 2025GR-3 Technical Reportcited by 110
  38. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos
  2. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy
  3. 2026Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data
  4. 2026LA4VLA: Learning to Act without Seeing via Language-Action Pretraining
  5. 2026ST-π: Structured SpatioTemporal VLA for Robotic Manipulation
  6. 2025ROSA: Harnessing Robot States for Vision-Language and Action Alignment
  7. 2026EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
  8. 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey