mlsanityData to 7 Oct 2026

Offline Reinforcement Learning for Visual Navigation

arXiv 2212.08244 ↗First posted 16 Dec 2022cs.RO cs.CV cs.LGcited by 28

On the map: Offline RL and imitation learning from data → Language-conditioned policies and VLAs

Abstract (arXiv)

Reinforcement learning can enable robots to navigate to distant goals while optimizing user-specified reward functions, including preferences for following lanes, staying on paved paths, or avoiding freshly mowed grass. However, online learning from trial-and-error for real-world robots is logistically challenging, and methods that instead can utilize existing datasets of robotic navigation data could be significantly more scalable and enable broader generalization. In this paper, we present ReViND, the first offline RL system for robotic navigation that can leverage previously collected data to optimize user-specified reward functions in the real-world. We evaluate our system for off-road navigation without any additional data collection or fine-tuning, and show that it can navigate to distant goals using only offline training from this dataset, and exhibit behaviors that qualitatively differ based on the user-specified reward function.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 48 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

30 of 48 references.

References · 30

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  10. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  11. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  12. 2018On Evaluation of Embodied Navigation Agentscited by 526
18 more
  1. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  2. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  3. 2020Learning to Explore using Active Neural SLAMcited by 308
  4. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  6. 2018Semi-parametric Topological Memory for Navigationcited by 203
  7. 2019Planning with Goal-Conditioned Policiescited by 144
  8. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 134
  9. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  10. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  11. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  12. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  13. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
  14. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  15. 2017PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planningcited by 64
  16. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  17. 2019gradSLAM: Automagically differentiable SLAMcited by 27
  18. 2018Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 7; 2025: 7; 2026: 5.

28 papers.

Sort:

By area:

  1. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  2. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  3. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  4. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  5. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  6. 2024Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RLcited by 21
  7. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  8. 2025TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Trackingcited by 16
  9. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 13
  10. 2023One-4-All: Neural Potential Fields for Embodied Navigationcited by 7
  11. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agentscited by 7
  12. 2024Learning Speed Adaptation for Flight in Cluttercited by 6
16 more
  1. 2024Scaling Vision-and-Language Navigation With Offline RLcited by 5
  2. 2024Feudal Networks for Visual Navigationcited by 4
  3. 2025Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Trainingcited by 4
  4. 2023Visual Semantic Navigation with Real Robotscited by 2
  5. 2024Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharingcited by 2
  6. 2025Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigationcited by 2
  7. 2023Mildly Constrained Evaluation Policy for Offline Reinforcement Learningcited by 1
  8. 2023Avoidance Navigation Based on Offline Pre-Training Reinforcement Learningcited by 1
  9. 2023VAPOR: Legged Robot Navigation in Outdoor Vegetation Using Offline Reinforcement Learningcited by 1
  10. 2024EnCoMP: Enhanced Covert Maneuver Planning with Adaptive Threat-Aware Visibility Estimation using Offline Reinforcement Learningcited by 1
  11. 2025Video-Enhanced Offline Reinforcement Learning: A Model-Based Approachcited by 1
  12. 2026Inference Time Policy Optimization for Offline RL with Differentiable World Modelscited by 1
  13. 2026Learning from Demonstration with Failure Awareness for Safe Robot Navigationcited by 1
  14. 2026FeudalNav: A Simple Framework for Visual Navigationcited by 0
  15. 2026120 Minutes and a Laptop: Minimalist Image-goal Navigation via Unsupervised Exploration and Offline RLcited by 0
  16. 2026CST-WM: A Causally Structured World Model for Embodied Visual Trackingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning
  2. 2021Offline Reinforcement Learning as Anti-Exploration
  3. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations
  4. 2020Offline Reinforcement Learning from Images with Latent Space Models
  5. 2024OffRIPP: Offline RL-based Informative Path Planning
  6. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning
  7. 2024Real-World Offline Reinforcement Learning from Vision Language Model Feedback
  8. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning