mlsanityData to 7 Oct 2026

GNM: A General Navigation Model to Drive Any Robot

arXiv 2210.03370 ↗First posted 7 Oct 2022cs.RO cs.AI cs.LGcited by 161

Not on the map. Why?

Cited by 22 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Learning provides a powerful tool for vision-based navigation, but the capabilities of learning-based policies are constrained by limited training data. If we could combine data from all available sources, including multiple kinds of robots, we could train more powerful navigation models. In this paper, we study how a general goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots, and enable broad generalization across environments and embodiments. We analyze the necessary design decisions for effective data sharing across robots, including the use of temporal context and standardized action spaces, and demonstrate that an omnipolicy trained from heterogeneous datasets outperforms policies trained on any single dataset. We curate 60 hours of navigation trajectories from 6 distinct robots, and deploy the trained GNM on a range of new robots, including an underactuated quadrotor. We find that training on diverse data leads to robustness against degradation in sensing and actuation. Using a pre-trained navigation model with broad generalization capabilities can bootstrap applications on novel robots going forward, and we hope that the GNM represents a step in that direction. For more information on the datasets, code, and videos, please check out our project page https://sites.google.com/view/drive-any-robot.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 7 Oct 2022.

Builds on

40 of 53 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  8. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  9. 2014How transferable are features in deep neural networks?cited by 1,216
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  11. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  12. 2018On Evaluation of Embodied Navigation Agentscited by 526
28 more
  1. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  2. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  3. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  4. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  5. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  6. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  7. 2018Semi-parametric Topological Memory for Navigationcited by 203
  8. 2020Neural Topological SLAM for Visual Navigationcited by 193
  9. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  10. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  11. 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 122
  12. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  13. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
  14. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  15. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 84
  16. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
  17. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  18. 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
  19. 2020Semantic Visual Navigation by Watching YouTube Videoscited by 60
  20. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
  21. 2019Deep Visual MPC-Policy Learning for Navigationcited by 54
  22. 2022TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Modelscited by 48
  23. 2019Contextual Imagined Goals for Self-Supervised Robotic Learningcited by 40
  24. 2019Scaling Local Control to Large-Scale Topological Navigationcited by 37
  25. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 36
  26. 2021NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challengecited by 32
  27. 2022Spatio-Temporal Graph Localization Networks for Image-based Navigationcited by 10
  28. 2021Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robotscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 16; 2024: 34; 2025: 50; 2026: 59.

161 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  4. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  6. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  7. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  8. 2024Navigation World Modelscited by 262
  9. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  10. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
  11. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  12. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
38 more
  1. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  2. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  3. 2023Large Language Models for Robotics: A Surveycited by 132
  4. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  5. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  6. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  7. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
  8. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  9. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  10. 2023SACSoN: Scalable Autonomous Control for Social Navigationcited by 60
  11. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  12. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
  13. 2023QUAR-VLA: Vision-Language-Action Model for Quadruped Robotscited by 55
  14. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
  15. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 46
  16. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  17. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  18. 2024Diffusion Transformer Policycited by 41
  19. 2024CityWalker: Learning Embodied Urban Navigation from Web-Scale Videoscited by 40
  20. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answeringcited by 37
  21. 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigationcited by 34
  22. 2024Learning Generalizable Feature Fields for Mobile Manipulationcited by 31
  23. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  24. 2025CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Modelscited by 28
  25. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 24
  26. 2025Prior Does Matter: Visual Navigation via Denoising Diffusion Bridge Modelscited by 24
  27. 2026Improved Baselines with Representation Autoencoderscited by 24
  28. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
  29. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 19
  30. 2022ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentationcited by 17
  31. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
  32. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  33. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  34. 2025RobotArena ∞: Scalable Robot Benchmarking via Real-to-Sim Translationcited by 15
  35. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 13
  36. 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuumcited by 13
  37. 2024REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environmentscited by 12
  38. 2025Learning to Drive Anywhere with Model-Based Reannotationcited by 12

50 most cited of 161 All 161 (JSON)