mlsanityData to 7 Oct 2026

PointVLA: Injecting the 3D World into Vision-Language-Action Models

arXiv 2503.07511 ↗First posted 10 Mar 2025cs.RO cs.CV cs.LGcited by 101

On the map: LLM reasoning → Spatial and embodied reasoning

also in 3 more
Abstract (arXiv)

Vision-Language-Action (VLA) models excel at robotic tasks by leveraging large-scale 2D vision-language pretraining, but their reliance on RGB images limits spatial reasoning critical for real-world interaction. Retraining these models with 3D data is computationally prohibitive, while discarding existing 2D datasets wastes valuable resources. To bridge this gap, we propose PointVLA, a framework that enhances pre-trained VLAs with point cloud inputs without requiring retraining. Our method freezes the vanilla action expert and injects 3D features via a lightweight modular block. To identify the most effective way of integrating point cloud representations, we conduct a skip-block analysis to pinpoint less useful blocks in the vanilla action expert, ensuring that 3D features are injected only into these blocks--minimizing disruption to pre-trained representations. Extensive experiments demonstrate that PointVLA outperforms state-of-the-art 2D imitation learning methods, such as OpenVLA, Diffusion Policy and DexVLA, across both simulated and real-world robotic tasks. Specifically, we highlight several key advantages of PointVLA enabled by point cloud integration: (1) Few-shot multi-tasking, where PointVLA successfully performs four different tasks using only 20 demonstrations each; (2) Real-vs-photo discrimination, where PointVLA distinguishes real objects from their images, leveraging 3D world knowledge to improve safety and reliability; (3) Height adaptability, Unlike conventional 2D imitation learning methods, PointVLA enables robots to adapt to objects at varying table height that unseen in train data. Furthermore, PointVLA achieves strong performance in long-horizon tasks, such as picking and packing objects from a moving conveyor belt, showcasing its ability to generalize across complex, dynamic environments.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 54 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

49 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul

Other references · 48

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  12. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
36 more
  1. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  2. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  3. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  4. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  5. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  6. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  7. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  8. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  9. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  10. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  11. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  12. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  13. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  14. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  15. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  16. 2024LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Modelcited by 128
  17. 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 112
  18. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  19. 2024Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inferencecited by 97
  20. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  21. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  22. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  23. 2024DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agentscited by 68
  24. 2025VLAS: Vision-Language-Action Model With Speech Instructions For Customized Robot Manipulationcited by 48
  25. 2024Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulationcited by 45
  26. 2025Humanoid-VLA: Towards Universal Humanoid Control with Visual Integrationcited by 43
  27. 2024RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effectivecited by 41
  28. 2024Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulationcited by 38
  29. 2021Sliced Recursive Transformercited by 26
  30. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  31. 2024Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Modelscited by 24
  32. 2024FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skippingcited by 23
  33. 2020Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priorscited by 21
  34. 2024Discrete Policy: Learning Disentangled Action Space for Multi-Task Robotic Manipulationcited by 17
  35. 2024DCPI-Depth: Explicitly Infusing Dense Correspondence Prior to Unsupervised Monocular Depth Estimationcited by 12
  36. 2024QUART-Online: Latency-Free Large Multimodal Language Model for Quadruped Robot Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 38; 2026: 63.

101 papers.

Sort:

By area:

  1. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  2. 2025ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulationcited by 87
  3. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  4. 2025BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Modelscited by 77
  5. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  6. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  7. 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
  8. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  9. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
  10. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  11. 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55
  12. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
38 more
  1. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  2. 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoningcited by 44
  3. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
  4. 2025FP3: A 3D Foundation Policy for Robotic Manipulationcited by 31
  5. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  6. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  7. 2025From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priorscited by 25
  8. 2026Adaptive Action Chunking at Inference-time for Vision-Language-Action Modelscited by 24
  9. 2025SwiftVLA: Unlocking Spatiotemporal Dynamics for Lightweight VLA Models at Minimal Overheadcited by 23
  10. 2025MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulationcited by 22
  11. 2025QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Modelscited by 19
  12. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 17
  13. 2025StereoVLA: Enhancing Vision-Language-Action Models with Stereo Visioncited by 16
  14. 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulationcited by 15
  15. 2025Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulationcited by 14
  16. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 14
  17. 2025CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Gamescited by 13
  18. 2025ManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulationcited by 13
  19. 2026Any3D-VLA: Enhancing VLA Robustness via Diverse Point Cloudscited by 13
  20. 2026Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Modelcited by 13
  21. 2025Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learningcited by 12
  22. 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulationcited by 12
  23. 2025GLaD: Geometric Latent Distillation for Vision-Language-Action Modelscited by 10
  24. 2026Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMscited by 10
  25. 2026Demystifying Action Space Design for Robotic Manipulation Policiescited by 9
  26. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  27. 2025OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulationcited by 7
  28. 2026VLA Knows Its Limits: Adaptive Execution Horizons for Robot Policiescited by 7
  29. 2026Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representationcited by 7
  30. 2026ConsisVLA-4D: Advancing Spatiotemporal Consistency in Efficient 3D-Perception and 4D-Reasoning for Robotic Manipulationcited by 6
  31. 2026PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interactioncited by 6
  32. 2026GEM: Generative Supervision Helps Embodied Intelligencecited by 6
  33. 2025EmbodiedMAE: A Unified 3D Multi-Modal Representation for Robot Manipulationcited by 5
  34. 2026UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Modelscited by 5
  35. 2026FocusVLA: Focused Visual Utilization for Vision-Language-Action Modelscited by 5
  36. 2026From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Modelcited by 5
  37. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  38. 2026E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenescited by 4

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos
  2. 20263DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding
  3. 2026GeomVLA: Unifying Scene, Motion, and Action in 3D
  4. 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Models
  5. 20243D-VLA: A 3D Vision-Language-Action Generative World Model
  6. 2026Bridge3D: Enabling Vision-Language-Action Models to See and Act in 3D
  7. 2026ST4VLA: Spatially Guided Training for Vision-Language-Action Models
  8. 2025OG-VLA: Orthographic Image Generation for 3D-Aware Vision-Language Action Model