mlsanityData to 7 Oct 2026

From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors

arXiv 2510.17439 ↗First posted 20 Oct 2025cs.RO cs.AI cs.CVcited by 25

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders, leaving a spatial reasoning gap that limits generalization and adaptability. Recent 3D integration techniques for VLAs either require specialized sensors and transfer poorly across modalities, or inject weak cues that lack geometry and degrade vision-language alignment. In this work, we introduce FALCON (From Spatial to Action), a novel paradigm that injects rich 3D spatial tokens into the action head. FALCON leverages spatial foundation models to deliver strong geometric priors from RGB alone, and includes an Embodied Spatial Model that can optionally fuse depth, or pose for higher fidelity when available, without retraining or architectural changes. To preserve language reasoning, spatial tokens are consumed by a Spatial-Enhanced Action Head rather than being concatenated into the vision-language backbone. These designs enable FALCON to address limitations in spatial representation, modality transferability, and alignment. In comprehensive evaluations across three simulation benchmarks and eleven real-world tasks, our proposed FALCON achieves state-of-the-art performance, consistently surpasses competitive baselines, and remains robust under clutter, spatial-prompt conditioning, and variations in object scale and height.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 51 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

44 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025PointVLA: Injecting the 3D World into Vision-Language-Action Models10 Mar
  2. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb

Other references · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  10. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  11. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  12. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
30 more
  1. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  2. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  3. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  4. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  6. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  7. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  8. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  9. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  10. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  11. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  12. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  13. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  14. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  15. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  16. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  17. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
  18. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
  19. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
  20. 2025GR-3 Technical Reportcited by 110
  21. 2025UP-VLA: A Unified Understanding and Prediction Model for Embodied Agentcited by 99
  22. 2024What Matters in Building Vision-Language-Action Models for Generalist Robotscited by 88
  23. 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
  24. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  25. 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55
  26. 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 49
  27. 2025Dense Policy: Bidirectional Autoregressive Learning of Actionscited by 19
  28. 2024DexGrasp-Diffusion: Diffusion-based Unified Functional Grasp Synthesis Method for Multi-Dexterous Robotic Handscited by 14
  29. 2025Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robotscited by 11
  30. 2025DSPv2: Improved Dense Policy for Effective and Generalizable Whole-body Mobile Manipulationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 24.

25 papers.

Sort:

By area:

  1. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
  2. 2026World Guidance: World Modeling in Condition Space for Action Generationcited by 32
  3. 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 30
  4. 2025Mixture of Horizons in Action Chunkingcited by 20
  5. 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulationcited by 15
  6. 2026SimVLA: A Simple VLA Baseline for Robotic Manipulationcited by 10
  7. 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 7
  8. 2026Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligencecited by 5
  9. 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Modelscited by 5
  10. 2026Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Modelscited by 4
  11. 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Modelcited by 3
  12. 2026From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Modelcited by 3
13 more
  1. 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Spacecited by 3
  2. 2026Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalizationcited by 2
  3. 2026PVI: Plug-in Visual Injection for Vision-Language-Action Modelscited by 1
  4. 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Drivingcited by 1
  5. 20263DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-trainingcited by 1
  6. 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Modelscited by 1
  7. 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulationcited by 1
  8. 2026OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RLcited by 0
  9. 2026X-Imitator: Spatial-Aware Imitation Learning via Bidirectional Action-Pose Interactioncited by 0
  10. 2026Breaking Structural Isolation: Scalable Graph Clustering via Community-Aware Sampling and Structural Entropycited by 0
  11. 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulationcited by 0
  12. 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulationcited by 0
  13. 2026StrataVLA: Hierarchical and Efficient 3D Geometric Grounding for Vision-Language-Action Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model
  2. 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning
  3. 2026VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models
  4. 2026GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model
  5. 2026ST4VLA: Spatially Guided Training for Vision-Language-Action Models
  6. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
  7. 2025Igniting VLMs toward the Embodied Space
  8. 2026Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models