# Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization

arXiv: https://arxiv.org/abs/2606.27663 · HTML: https://mlsanity.com/paper/2606.27663/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 26 of its 27 references that are in the map.
- First posted on arXiv 26 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- Vision-language-action models / [3D and spatial-aware VLA](https://mlsanity.com/vla/3d-and-spatial-aware-vla/)

## Builds on (most cited first, 26 of 26)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model](https://mlsanity.com/paper/2501.15830/) (2025)
- [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Robotic Control via Embodied Chain-of-Thought Reasoning](https://mlsanity.com/paper/2407.08693/) (2024)
- [3D-VLA: A 3D Vision-Language-Action Generative World Model](https://mlsanity.com/paper/2403.09631/) (2024)
- [LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models](https://mlsanity.com/paper/2510.13626/) (2025)
- [RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics](https://mlsanity.com/paper/2406.10721/) (2024)
- [LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization](https://mlsanity.com/paper/2510.03827/) (2025)
- [HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation](https://mlsanity.com/paper/2502.05485/) (2025)
- [MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting](https://mlsanity.com/paper/2403.03174/) (2024)
- [GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data](https://mlsanity.com/paper/2505.03233/) (2025)
- [PointVLA: Injecting the 3D World into Vision-Language-Action Models](https://mlsanity.com/paper/2503.07511/) (2025)
- [RoboGround: Robotic Manipulation with Grounded Vision-Language Priors](https://mlsanity.com/paper/2504.21530/) (2025)
- [CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordance](https://mlsanity.com/paper/2412.20451/) (2024)
- [From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors](https://mlsanity.com/paper/2510.17439/) (2025)
- [Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control](https://mlsanity.com/paper/2602.13193/) (2026)
- [VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models](https://mlsanity.com/paper/2603.22003/) (2026)

## Built on by (most cited first, 2 of 2)

- [LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models](https://mlsanity.com/paper/2609.05178/) (2026)
- [Correcting WHERE, Preserving HOW: Compositional Generalization for Vision-Language-Action Models via Referential Guidance](https://mlsanity.com/paper/2609.38616/) (2026)

## Abstract (arXiv)

Vision-Language-Action (VLA) models leverage large-scale vision-language pretraining for flexible robot manipulation, yet at test time they remain brittle to changed object positions and to familiar scenes paired with different instructions. A growing family of methods addresses this brittleness by supplying the policy with grounding signals, such as 2D pixel coordinates for object localization and placement. However, we find that how the grounding signal is represented and injected matters more than the signal itself. In this work, we propose a lightweight module that represents the grounding signal in 3D and injects the resulting embedding directly into the action head. The module is a two-layer MLP and requires no changes to the VLA backbone or pretraining pipeline, yet it yields substantially larger gains than language- or visual-prompting alternatives. On LIBERO-PRO, our method improves the average success rate of GR00T-N1.6 from 31.2 to 77.5 under task perturbation and from 28.1 to 60.2 under position perturbation. Comparable gains are also achieved for π0.5, demonstrating that the mechanism is backbone-agnostic across VLAs with diffusion-based action heads. We further validate the practical applicability with real-world experiments. Together, these results support our central finding: lifting adequate 2D grounding into 3D and injecting it into the action head enables spatial and instance-level task generalization in VLAs.