# OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation

arXiv: https://arxiv.org/abs/2509.19480 · HTML: https://mlsanity.com/paper/2509.19480/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models.
- Builds on 39 of its 49 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Placed in

- Vision-language-action models / [Generalist VLA models and benchmarks](https://mlsanity.com/vla/generalist-vla-models-and-benchmarks/)

## Builds on (most cited first, 30 of 39)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics](https://mlsanity.com/paper/2506.01844/) (2025)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models](https://mlsanity.com/paper/2402.07865/) (2024)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [Learning Latent Plans from Play](https://mlsanity.com/paper/1903.01973/) (2019)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation](https://mlsanity.com/paper/2203.10421/) (2022)
- [GNM: A General Navigation Model to Drive Any Robot](https://mlsanity.com/paper/2210.03370/) (2022)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2204.06252/) (2022)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Rapid Exploration for Open-World Navigation with Latent Goal Models](https://mlsanity.com/paper/2104.05859/) (2021)
- [Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2203.12667/) (2022)
- [Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation](https://mlsanity.com/paper/2408.11812/) (2024)
- [Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation](https://mlsanity.com/paper/1709.10489/) (2017)
- [Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs](https://mlsanity.com/paper/2407.07775/) (2024)
- [SACSoN: Scalable Autonomous Control for Social Navigation](https://mlsanity.com/paper/2306.01874/) (2023)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [Deep Visual MPC-Policy Learning for Navigation](https://mlsanity.com/paper/1903.02749/) (2019)

## Built on by (most cited first, 30 of 38)

- [Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control](https://mlsanity.com/paper/2602.13193/) (2026)
- [LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfer](https://mlsanity.com/paper/2602.10556/) (2026)
- [Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation](https://mlsanity.com/paper/2606.15869/) (2026)
- [TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments](https://mlsanity.com/paper/2602.02459/) (2026)
- [AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge](https://mlsanity.com/paper/2602.13476/) (2026)
- [An Efficient and Multi-Modal Navigation System with One-Step World Model](https://mlsanity.com/paper/2601.12277/) (2026)
- [RAE-NWM: Navigation World Model in Dense Visual Representation Space](https://mlsanity.com/paper/2603.09241/) (2026)
- [Do Audio-Visual Large Language Models Really See and Hear?](https://mlsanity.com/paper/2604.02605/) (2026)
- [RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learning](https://mlsanity.com/paper/2602.18742/) (2026)
- [NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation](https://mlsanity.com/paper/2606.13494/) (2026)
- [VLA-R: Vision-Language Action Retrieval toward Open-World End-to-End Autonomous Driving](https://mlsanity.com/paper/2511.12405/) (2025)
- [Safe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulation](https://mlsanity.com/paper/2603.05754/) (2026)
- [Latent World Models with Monotone Planning Costs for Image-Goal Navigation](https://mlsanity.com/paper/2608.09073/) (2026)
- [MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning](https://mlsanity.com/paper/2510.03142/) (2025)
- [UrbanVLA: A Vision-Language-Action Model for Urban Micromobility](https://mlsanity.com/paper/2510.23576/) (2025)
- [Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement](https://mlsanity.com/paper/2606.18247/) (2026)
- [AsyncShield: A Plug-and-Play Edge Adapter for Asynchronous Cloud-based VLA Navigation](https://mlsanity.com/paper/2604.24086/) (2026)
- [Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation](https://mlsanity.com/paper/2603.19166/) (2026)
- [Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies](https://mlsanity.com/paper/2607.05122/) (2026)
- [A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation](https://mlsanity.com/paper/2607.09792/) (2026)
- [TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model](https://mlsanity.com/paper/2609.09158/) (2026)
- [Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder](https://mlsanity.com/paper/2603.05528/) (2026)
- [APPLV: Adaptive Planner Parameter Learning from Vision-Language-Action Model](https://mlsanity.com/paper/2603.08862/) (2026)
- [Act on What You See: Unlocking Safe Social Navigation in Vision-Language-Action Models](https://mlsanity.com/paper/2606.10495/) (2026)
- [Foresight: Iterative Reasoning About Clues that Matter for Navigation](https://mlsanity.com/paper/2606.12550/) (2026)
- [Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control](https://mlsanity.com/paper/2606.28760/) (2026)
- [GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model](https://mlsanity.com/paper/2607.06882/) (2026)
- [CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adapting](https://mlsanity.com/paper/2608.06688/) (2026)
- [If, Then, Otherwise: Diagnosing Conditional Branching in Vision-Language Navigation](https://mlsanity.com/paper/2608.17318/) (2026)
- [ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models](https://mlsanity.com/paper/2609.13231/) (2026)

## Abstract (arXiv)

Humans can flexibly interpret and compose different goal specifications, such as language instructions, spatial coordinates, or visual references, when navigating to a destination. In contrast, most existing robotic navigation policies are trained on a single modality, limiting their adaptability to real-world scenarios where different forms of goal specification are natural and complementary. In this work, we present a training framework for robotic foundation models that enables omni-modal goal conditioning for vision-based navigation. Our approach leverages a high-capacity vision-language-action (VLA) backbone and trains with three primary goal modalities: 2D poses, egocentric images, and natural language, as well as their combinations, through a randomized modality fusion strategy. This design not only expands the pool of usable datasets but also encourages the policy to develop richer geometric, semantic, and visual representations. The resulting model, OmniVLA, achieves strong generalization to unseen environments, robustness to scarce modalities, and the ability to follow novel natural language instructions. We demonstrate that OmniVLA outperforms specialist baselines across modalities and offers a flexible foundation for fine-tuning to new modalities and tasks. We believe OmniVLA provides a step toward broadly generalizable and flexible navigation policies, and a scalable path for building omni-modal robotic foundation models. We present videos showcasing OmniVLA performance and will release its checkpoints and training code on our project page.