# Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies

arXiv: https://arxiv.org/abs/2607.05122 · HTML: https://mlsanity.com/paper/2607.05122/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in 3D and spatial-aware VLA, one of 13 directions in Vision-language-action models.
- Builds on 17 of its 20 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Placed in

- Vision-language-action models / [3D and spatial-aware VLA](https://mlsanity.com/vla/3d-and-spatial-aware-vla/)

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Language Is Not All You Need: Aligning Perception with Language Models](https://mlsanity.com/paper/2302.14045/) (2023)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs](https://mlsanity.com/paper/2402.07872/) (2024)
- [A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS](https://mlsanity.com/paper/2304.00501/) (2023)
- [Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs](https://mlsanity.com/paper/2407.07775/) (2024)
- [OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation](https://mlsanity.com/paper/2509.19480/) (2025)
- [Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust](https://mlsanity.com/paper/2410.01971/) (2024)
- [CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models](https://mlsanity.com/paper/2508.13446/) (2025)
- [Contrastive Variational Autoencoder Enhances Salient Features](https://mlsanity.com/paper/1902.04601/) (2019)
- [VL-TGS: Trajectory Generation and Selection using Vision Language Models in Mapless Outdoor Environments](https://mlsanity.com/paper/2408.02454/) (2024)
- [GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation](https://mlsanity.com/paper/2602.18164/) (2026)

## Built on by (most cited first, 1 of 1)

- [Seeing the Invisible: Physics-Guided Visual Prompting for Temperature- and Radiation-Aware VLA Navigation](https://mlsanity.com/paper/2610.07558/) (2026)

## Abstract (arXiv)

Vision-language-action (VLA) models enable robot navigation from natural language and visual goals, but remain susceptible to perceptual distractions and ambiguous scene interpretations. This paper presents the first empirical evaluation of visual grounding for VLA navigation policies. We propose a real-time segmentation-based grounding method that highlights traversable areas in green and non-traversable areas in red using SegFormer. Two variants are evaluated: observation-only segmentation and joint observation-goal augmentation. Using OmniVLA on the Grand Tour dataset, we show that visual grounding reduces the mean waypoint error by 27-44% at the farthest waypoint, depending on the instruction length. The benefits are greater for long instructions than for short instructions, and grounding provides little improvement for image goals. Normalized error analysis indicates that grounding primarily acts as a trajectory length regularizer, reducing the predicted path length by 30% without improving per-unit-distance reasoning. Our results indicate that visual grounding offers a simple, computationally inexpensive method to improve VLA navigation without model retraining, although it cannot compensate for missing training signals in out-of-distribution instructions.