Seeing Space and Motion: Enhancing Latent Actions with Geometric and Dynamic Awareness for Vision-Language-Action Models
On the map: Vision-language-action models → Human-video and latent-action pretraining
Abstract (arXiv)
Latent Action Models (LAMs) enable Vision- Language-Action (VLA) systems to learn semantic action representations from large-scale unannotated data. Yet, we identify two bottlenecks of LAMs: 1) the commonly adopted end-to-end trained image encoder suffers from poor spatial understanding; 2) LAMs can be fragile when input frames are temporally distant, leading to limited temporal percep- tion. Such factors inevitably hinder stable and clear action modeling. To this end, we propose Farsighted-LAM, a latent action framework with geometry-aware spatial encoding and multi-scale temporal modeling, capturing structural priors and dynamic motion patterns from consecutive frames. We further propose SSM-VLA, an end-to-end VLA framework built upon Farsighted-LAM, which integrates structured perception with a visual Chain-of-Thought module to explicitly reason about environmental dynamics, enhancing decision consistency and interpretability. We validate SSM-VLA on multiple VLA tasks in both simulation and real-world settings, and achieve state-of- the-art performance. Our results demonstrate that our strategy of combining geometry-aware modeling, temporal coherence, and explicit reasoning is effective in enhancing the robustness and generalizability of embodied intelligence.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. Builds on 36 of its 42 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
36 of 42 references.
Closest ancestors · 3
Same direction, newest first
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actions9 May
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations19 Dec
- 2024Latent Action Pretraining from Videos15 Oct
Other references · 33
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2024Depth Anything V2cited by 1,422
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2022A Generalist Agentcited by 808
- 2024Genie: Generative Interactive Environmentscited by 747
21 more
- 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
- 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
- 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
- 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 213
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
- 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 161
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 153
- 2024DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshingcited by 138
- 2025UP-VLA: A Unified Understanding and Prediction Model for Embodied Agentcited by 102
- 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 82
- 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 74
- 2024Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulationcited by 73
- 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AIcited by 72
- 2024Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videoscited by 69
- 2024Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulationcited by 53
- 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 44
- 2025VideoWorld: Exploring Knowledge Learning from Unlabeled Videoscited by 43
- 2024VideoAgent: Self-Improving Video Generationcited by 25
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models
- 2026LARA: Latent Action Representation Alignment for Vision-Language-Action Models
- 2026Chain of World: World Model Thinking in Latent Motion
- 2026ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- 2026PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space
- 2026Latent Cluster Analysis for Vision-Language-Action Models
- 2026Vision-Language Models Unlock Task-Centric Latent Actions
- 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations