# Embodied Multimodal Grounding for Open-Vocabulary Mobile Manipulation via Semantic 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2608.10756 · HTML: https://mlsanity.com/paper/2608.10756/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in 3D and spatial-aware VLA, one of 13 directions in Vision-language-action models.
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Placed in

- Vision-language-action models / [3D and spatial-aware VLA](https://mlsanity.com/vla/3d-and-spatial-aware-vla/)

## Builds on (most cited first, 12 of 12)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control](https://mlsanity.com/paper/2502.05855/) (2025)
- [GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping](https://mlsanity.com/paper/2403.09637/) (2024)
- [Visual Whole-Body Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2403.16967/) (2024)
- [ASC: Adaptive Skill Coordination for Robotic Mobile Manipulation](https://mlsanity.com/paper/2304.00410/) (2023)
- [Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion](https://mlsanity.com/paper/2303.11330/) (2023)
- [Pedipulate: Enabling Manipulation Skills using a Quadruped Robot's Leg](https://mlsanity.com/paper/2402.10837/) (2024)
- [GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion](https://mlsanity.com/paper/2309.15459/) (2023)
- [Learning Force Control for Legged Manipulation](https://mlsanity.com/paper/2405.01402/) (2024)
- [RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulation](https://mlsanity.com/paper/2403.17367/) (2024)
- [Whole-body MPC for highly redundant legged manipulators: experimental evaluation with a 37 DoF dual-arm quadruped](https://mlsanity.com/paper/2310.02907/) (2023)

## Abstract (arXiv)

Embodied mobile manipulation requires language, visual observations, three-dimensional scene structure, and action feasibility to be aligned before execution. We study open-vocabulary target grounding with few-shot manipulation in local household workspaces and present an embodied multimodal grounding framework that integrates active multi-view Semantic 3D Gaussian Splatting (Semantic-3DGS), reachability-aware base positioning, and a diffusion-based vision-language-action policy. A task-driven local Semantic-3DGS serves as a shared interface across active sensing, language-conditioned 3D localization, obstacle-aware scene reasoning, base preparation, and semantic conditioning of the action model. To preserve pretrained action priors, the 3D semantic cues are injected only into the late action-expert blocks. In expanded 50-trial real-robot evaluations against representative vision-language-action (VLA) approaches, the full system achieves 60% long-horizon success compared with 40% for PointVLA and 28% for DexVLA, and reaches 74% success in heavily cluttered manipulation compared with 52% for the single-view variant and 46% for PointVLA. It also maintains 75% success under a 75 cm height shift and eliminates photo-induced false grasps. These results indicate that explicit, refreshable 3D semantic grounding can improve robustness under clutter, occlusion, viewpoint variation, and embodiment constraints.