Learning to Drive Anywhere with Model-Based Reannotation
Not on the map. Why?
Abstract (arXiv)
Developing broadly generalizable visual navigation policies for robots is a significant challenge, primarily constrained by the availability of large-scale, diverse training data. While curated datasets collected by researchers offer high quality, their limited size restricts policy generalization. To overcome this, we explore leveraging abundant, passively collected data sources, including large volumes of crowd-sourced teleoperation data and unlabeled YouTube videos, despite their potential for lower quality or missing action labels. We propose Model-Based ReAnnotation (MBRA), a framework that utilizes a learned short-horizon, model-based expert model to relabel or generate high-quality actions for these passive datasets. This relabeled data is then distilled into LogoNav, a long-horizon navigation policy conditioned on visual goals or GPS waypoints. We demonstrate that LogoNav, trained using MBRA-processed data, achieves state-of-the-art performance, enabling robust navigation over distances exceeding 300 meters in previously unseen indoor and outdoor environments. Our extensive real-world evaluations, conducted across a fleet of robots (including quadrupeds) in six cities on three continents, validate the policy's ability to generalize and navigate effectively even amidst pedestrians in crowded settings.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 8 May 2025.
Builds on
35 of 53 references.
References · 35
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
- 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
- 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
23 more
- 2023ViNT: A Foundation Model for Visual Navigationcited by 222
- 2018Semi-parametric Topological Memory for Navigationcited by 209
- 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
- 2022Deep Patch Visual Odometrycited by 190
- 2022GNM: A General Navigation Model to Drive Any Robotcited by 171
- 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 127
- 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
- 2018Zero-Shot Visual Imitationcited by 117
- 2017Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigationcited by 77
- 2023SACSoN: Scalable Autonomous Control for Social Navigationcited by 61
- 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 58
- 2019Deep Visual MPC-Policy Learning for Navigationcited by 56
- 2022TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Modelscited by 49
- 2024CityWalker: Learning Embodied Urban Navigation from Web-Scale Videoscited by 41
- 2019Scaling Local Control to Large-Scale Topological Navigationcited by 37
- 2024LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videoscited by 32
- 2018GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimationcited by 29
- 2022ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentationcited by 19
- 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
- 2022Spatio-Temporal Graph Localization Networks for Image-based Navigationcited by 10
- 2020Robot Perception of Static and Dynamic Objects with an Autonomous Floor Scrubbercited by 6
- 2021Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Modelcited by 4
- 2022Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approachcited by 1
Built on by
Citing papers per year: 2025: 6; 2026: 7.
13 papers.
Sort:
By area:
- 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigationcited by 38
- 2025From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learningcited by 12
- 2025SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigationcited by 12
- 2026AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edgecited by 10
- 2025UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videoscited by 5
- 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
- 2025Wanderland: Geometrically Grounded Simulation for Open-World Embodied AIcited by 3
- 2026Learning Sidewalk Autopilot from Multi-Scale Imitation with Corrective Behavior Expansioncited by 1
- 2026From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigationcited by 0
- 2026EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awarenesscited by 0
- 2026CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adaptingcited by 0
- 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0
1 more
- 2026Training Generalist Navigation Policies via Action Self-Distillation13 Oct 2026
- 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform1 Oct 2026
- 2026CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adapting7 Aug 2026
- 2026EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awareness22 Jul 2026
- 2026From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation10 Jun 2026
- 2026Learning Sidewalk Autopilot from Multi-Scale Imitation with Corrective Behavior Expansion23 Mar 2026
- 2026AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge13 Feb 2026
- 2025SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigation26 Nov 2025
- 2025Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI25 Nov 2025
- 2025UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos16 Oct 2025
- 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning3 Oct 2025
- 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation23 Sep 2025
- 2025SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigationcited by 12
- 2026AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edgecited by 10
- 2025UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videoscited by 5
- 2025Wanderland: Geometrically Grounded Simulation for Open-World Embodied AIcited by 3
- 2026From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigationcited by 0
- 2026EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awarenesscited by 0
- 2026EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awareness22 Jul 2026
- 2026From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation10 Jun 2026
- 2026AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge13 Feb 2026
- 2025SocialNav: Training Human-Inspired Foundation Model for Socially-Aware Embodied Navigation26 Nov 2025
- 2025Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI25 Nov 2025
- 2025UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos16 Oct 2025