From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors
On the map: LLM reasoning → Spatial and embodied reasoning
also in 1 more
Abstract (arXiv)
Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders, leaving a spatial reasoning gap that limits generalization and adaptability. Recent 3D integration techniques for VLAs either require specialized sensors and transfer poorly across modalities, or inject weak cues that lack geometry and degrade vision-language alignment. In this work, we introduce FALCON (From Spatial to Action), a novel paradigm that injects rich 3D spatial tokens into the action head. FALCON leverages spatial foundation models to deliver strong geometric priors from RGB alone, and includes an Embodied Spatial Model that can optionally fuse depth, or pose for higher fidelity when available, without retraining or architectural changes. To preserve language reasoning, spatial tokens are consumed by a Spatial-Enhanced Action Head rather than being concatenated into the vision-language backbone. These designs enable FALCON to address limitations in spatial representation, modality transferability, and alignment. In comprehensive evaluations across three simulation benchmarks and eleven real-world tasks, our proposed FALCON achieves state-of-the-art performance, consistently surpasses competitive baselines, and remains robust under clutter, spatial-prompt conditioning, and variations in object scale and height.
Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 51 references that are in the map. First posted on arXiv 20 Oct 2025.
Builds on
44 of 51 references.
Closest ancestors · 2
Same direction, newest first
- 2025PointVLA: Injecting the 3D World into Vision-Language-Action Models10 Mar
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
Other references · 42
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
30 more
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
- 2024Grounding Image Matching in 3D with MASt3Rcited by 960
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2025Continuous 3D Perception Model with Persistent Statecited by 527
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
- 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
- 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
- 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
- 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
- 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
- 2025GR-3 Technical Reportcited by 110
- 2025UP-VLA: A Unified Understanding and Prediction Model for Embodied Agentcited by 99
- 2024What Matters in Building Vision-Language-Action Models for Generalist Robotscited by 88
- 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
- 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
- 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55
- 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 49
- 2025Dense Policy: Bidirectional Autoregressive Learning of Actionscited by 19
- 2024DexGrasp-Diffusion: Diffusion-based Unified Functional Grasp Synthesis Method for Multi-Dexterous Robotic Handscited by 14
- 2025Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robotscited by 11
- 2025DSPv2: Improved Dense Policy for Effective and Generalizable Whole-body Mobile Manipulationcited by 8
Built on by
Citing papers per year: 2025: 1; 2026: 24.
25 papers.
Sort:
By area:
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
- 2026World Guidance: World Modeling in Condition Space for Action Generationcited by 32
- 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 30
- 2025Mixture of Horizons in Action Chunkingcited by 20
- 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulationcited by 15
- 2026SimVLA: A Simple VLA Baseline for Robotic Manipulationcited by 10
- 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 7
- 2026Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligencecited by 5
- 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Modelscited by 5
- 2026Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Modelscited by 4
- 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Modelcited by 3
- 2026From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Modelcited by 3
13 more
- 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Spacecited by 3
- 2026Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalizationcited by 2
- 2026PVI: Plug-in Visual Injection for Vision-Language-Action Modelscited by 1
- 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Drivingcited by 1
- 20263DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-trainingcited by 1
- 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Modelscited by 1
- 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulationcited by 1
- 2026OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RLcited by 0
- 2026X-Imitator: Spatial-Aware Imitation Learning via Bidirectional Action-Pose Interactioncited by 0
- 2026Breaking Structural Isolation: Scalable Graph Clustering via Community-Aware Sampling and Structural Entropycited by 0
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulationcited by 0
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulationcited by 0
- 2026StrataVLA: Hierarchical and Efficient 3D Geometric Grounding for Vision-Language-Action Modelscited by 0
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation27 Aug 2026
- 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation5 Aug 2026
- 2026StrataVLA: Hierarchical and Efficient 3D Geometric Grounding for Vision-Language-Action Models3 Aug 2026
- 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space2 Aug 2026
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation14 Jul 2026
- 2026Breaking Structural Isolation: Scalable Graph Clustering via Community-Aware Sampling and Structural Entropy6 Jul 2026
- 2026From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model6 Jul 2026
- 2026Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization26 Jun 2026
- 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models9 Jun 2026
- 20263DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training3 Jun 2026
- 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Driving29 May 2026
- 2026Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models23 May 2026
13 more
- 2026X-Imitator: Spatial-Aware Imitation Learning via Bidirectional Action-Pose Interaction12 May 2026
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr 2026
- 2026OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RL20 Apr 2026
- 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Models30 Mar 2026
- 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model19 Mar 2026
- 2026PVI: Plug-in Visual Injection for Vision-Language-Action Models13 Mar 2026
- 2026Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence8 Mar 2026
- 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models6 Mar 2026
- 2026World Guidance: World Modeling in Condition Space for Action Generation25 Feb 2026
- 2026SimVLA: A Simple VLA Baseline for Robotic Manipulation20 Feb 2026
- 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulation2 Feb 2026
- 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models16 Jan 2026
- 2025Mixture of Horizons in Action Chunking24 Nov 2025
- 2025Mixture of Horizons in Action Chunkingcited by 20
- 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulationcited by 15
- 2026SimVLA: A Simple VLA Baseline for Robotic Manipulationcited by 10
- 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 7
- 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Modelscited by 5
- 2026Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Modelscited by 4
- 2026From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Modelcited by 3
- 2026PVI: Plug-in Visual Injection for Vision-Language-Action Modelscited by 1
- 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Modelscited by 1
- 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulationcited by 1
- 2026OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RLcited by 0
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulationcited by 0
12 most cited of 13 All 13 (JSON)
- 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation27 Aug 2026
- 2026World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation5 Aug 2026
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation14 Jul 2026
- 2026From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model6 Jul 2026
- 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models9 Jun 2026
- 2026Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models23 May 2026
- 2026OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RL20 Apr 2026
- 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Models30 Mar 2026
- 2026PVI: Plug-in Visual Injection for Vision-Language-Action Models13 Mar 2026
- 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models6 Mar 2026
- 2026SimVLA: A Simple VLA Baseline for Robotic Manipulation20 Feb 2026
- 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulation2 Feb 2026
12 newest of 13 All 13 (JSON)
- 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 30
- 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Modelcited by 3
- 20263DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-trainingcited by 1
- 20263DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training3 Jun 2026
- 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model19 Mar 2026
- 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models16 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model
- 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning
- 2026VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models
- 2026GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model
- 2026ST4VLA: Spatially Guided Training for Vision-Language-Action Models
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
- 2025Igniting VLMs toward the Embodied Space
- 2026Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models