mlsanityData to 14 Oct 2026

Learning to Drive Anywhere with Model-Based Reannotation

arXiv 2505.05592 ↗First posted 8 May 2025cs.RO cs.CV cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Developing broadly generalizable visual navigation policies for robots is a significant challenge, primarily constrained by the availability of large-scale, diverse training data. While curated datasets collected by researchers offer high quality, their limited size restricts policy generalization. To overcome this, we explore leveraging abundant, passively collected data sources, including large volumes of crowd-sourced teleoperation data and unlabeled YouTube videos, despite their potential for lower quality or missing action labels. We propose Model-Based ReAnnotation (MBRA), a framework that utilizes a learned short-horizon, model-based expert model to relabel or generate high-quality actions for these passive datasets. This relabeled data is then distilled into LogoNav, a long-horizon navigation policy conditioned on visual goals or GPS waypoints. We demonstrate that LogoNav, trained using MBRA-processed data, achieves state-of-the-art performance, enabling robust navigation over distances exceeding 300 meters in previously unseen indoor and outdoor environments. Our extensive real-world evaluations, conducted across a fleet of robots (including quadrupeds) in six cities on three continents, validate the policy's ability to generalize and navigate effectively even amidst pedestrians in crowded settings.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

35 of 53 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023Segment Anythingcited by 8,234
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  7. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  8. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  9. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  10. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  11. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  12. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
23 more
  1. 2023ViNT: A Foundation Model for Visual Navigationcited by 222
  2. 2018Semi-parametric Topological Memory for Navigationcited by 209
  3. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
  4. 2022Deep Patch Visual Odometrycited by 190
  5. 2022GNM: A General Navigation Model to Drive Any Robotcited by 171
  6. 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 127
  7. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
  8. 2018Zero-Shot Visual Imitationcited by 117
  9. 2017Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigationcited by 77
  10. 2023SACSoN: Scalable Autonomous Control for Social Navigationcited by 61
  11. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 58
  12. 2019Deep Visual MPC-Policy Learning for Navigationcited by 56
  13. 2022TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Modelscited by 49
  14. 2024CityWalker: Learning Embodied Urban Navigation from Web-Scale Videoscited by 41
  15. 2019Scaling Local Control to Large-Scale Topological Navigationcited by 37
  16. 2024LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videoscited by 32
  17. 2018GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimationcited by 29
  18. 2022ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentationcited by 19
  19. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
  20. 2022Spatio-Temporal Graph Localization Networks for Image-based Navigationcited by 10
  21. 2020Robot Perception of Static and Dynamic Objects with an Autonomous Floor Scrubbercited by 6
  22. 2021Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Modelcited by 4
  23. 2022Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approachcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area: