3D-Mix for VLA: A Plug-and-Play Module for Integrating VGGT-based 3D Information into Vision-Language-Action Models
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Vision-Language-Action (VLA) models leverage Multimodal Large Language Models (MLLMs) for robotic control, but recent studies reveal that MLLMs exhibit limited spatial intelligence due to training predominantly on 2D data, resulting in inadequate 3D perception for manipulation tasks. While recent approaches incorporate specialized 3D vision models such as VGGT to enhance spatial understanding, they employ diverse integration mechanisms without systematic investigation, leaving the optimal fusion strategy unclear. We conduct a comprehensive pilot study comparing nine VGGT integration schemes on standardized benchmarks and find that semantic-conditioned gated fusion, which adaptively balances 2D semantic and 3D geometric features based on task context, achieved the strongest performance among all nine evaluated fusion schemes in our pilot study. We present 3D-Mix, a plug-and-play module that integrates into diverse VLA architectures (GR00T-style and π-style) without modifying existing MLLM or action expert components. Experiments across six MLLM series (nine model variants, 2B--8B parameters) on SIMPLER and LIBERO show that 3D-Mix delivers consistent performance gains, averaging +7.0% on the out-of-domain (OOD) SIMPLER benchmark across all nine GR00T-style variants, establishing a principled approach for enhancing spatial intelligence in VLA systems.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 13 references that are in the map. First posted on arXiv 25 Mar 2026.
Builds on
8 of 13 references.
References · 8
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
- 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
- 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
- 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55
- 2025SpatialTree: How Spatial Abilities Branch Out in MLLMscited by 11
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Modelcited by 13
- 2026IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulationcited by 7
- 2026Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulationcited by 3
- 2026Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3Rcited by 2
- 2026Revisiting Local Context for Long-Horizon Streaming 3D Reconstructioncited by 0
- 2026Spatial Grafting: Grounding 3D Features for Flow-Matching Robot Policiescited by 0
- 2026Spatial Grafting: Grounding 3D Features for Flow-Matching Robot Policies28 Sep 2026
- 2026Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction27 Aug 2026
- 2026Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R19 May 2026
- 2026Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model14 May 2026
- 2026IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation14 May 2026
- 2026Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation12 May 2026
Similar, not cited
Close in wording, no citation either way
- 20263DVLA: Enhancing Vision-Language-Action Models via 3D Spatial and Instance Understanding
- 2026Bridge3D: Enabling Vision-Language-Action Models to See and Act in 3D
- 2026AugVLA-3D: Depth-Driven Feature Augmentation for Vision-Language-Action Models
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation
- 20253D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Tasks
- 2026V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models
- 2026BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
- 2026FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision-Language-Action Models