mlsanityData to 14 Oct 2026

VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation

arXiv 2105.09371 ↗First posted 19 May 2021cs.RO cs.AI cs.CVcited by 31

On the map: Offline RL and imitation learning from data → Language-conditioned policies and VLAs

Abstract (arXiv)

While imitation learning for vision based autonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or worse yet the demonstrator does not have access to the platform at all? Is imitation learning for vision based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent ideas from the Imitation from Observation (IfO) literature can be brought to bear such that a robot can learn to navigate using only ego centric video collected by a demonstrator, even in the presence of viewpoint mismatch. To this end, we introduce a new algorithm, Visual Observation only Imitation Learning for Autonomous navigation (VOILA), that can successfully learn navigation policies from a single video demonstration collected from a physically different agent. We evaluate VOILA in the photorealistic AirSim simulator and show that VOILA not only successfully imitates the expert, but that it also learns navigation policies that can generalize to novel environments. Further, we demonstrate the effectiveness of VOILA in a real world setting by showing that it allows a wheeled Jackal robot to successfully imitate a human walking in an environment using a video recorded using a mobile phone camera.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 34 references that are in the map. First posted on arXiv 19 May 2021.

Builds on

20 of 34 references.

References · 20

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  3. 2016End to End Learning for Self-Driving Carscited by 1,009
  4. 2015NetVLAD: CNN architecture for weakly supervised place recognitioncited by 941
  5. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  6. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  7. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  8. 2018Learning to Drive in a Daycited by 273
  9. 2018Generative Adversarial Imitation from Observationcited by 184
  10. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  11. 2019From Variational to Deterministic Autoencoderscited by 133
  12. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
8 more
  1. 2019Recent Advances in Imitation Learning from Observationcited by 93
  2. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 70
  3. 2020Semantic Visual Navigation by Watching YouTube Videoscited by 60
  4. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  5. 2019Imitation Learning from Video by Leveraging Proprioceptioncited by 23
  6. 2019RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstrationcited by 19
  7. 2019Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigationcited by 13
  8. 2018Navigation by Imitation in a Pedestrian-Rich Environmentcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 10; 2024: 9; 2025: 2; 2026: 4.

31 papers.

Sort:

By area:

  1. 2022Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigationcited by 127
  2. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 75
  3. 2021Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulationcited by 59
  4. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  5. 2022K-VIL: Keypoints-based Visual Imitation Learningcited by 27
  6. 2022VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamicscited by 21
  7. 2023CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Groundingcited by 13
  8. 2022High-Speed Accurate Robot Control using Learned Forward Kinodynamics and Non-linear Least Squares Optimizationcited by 11
  9. 2023STERLING: Self-Supervised Terrain Representation Learning from Unconstrained Robot Experiencecited by 11
  10. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  11. 2025Spatially-Enhanced Recurrent Memory for Long-Range Mapless Navigation via End-to-End Reinforcement Learningcited by 11
  12. 2022Adversarial Imitation Learning from Video using a State Observercited by 10
19 more
  1. 2023Wait, That Feels Familiar: Learning to Extrapolate Human Preferences for Preference Aligned Path Planningcited by 7
  2. 2024Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terraincited by 7
  3. 2023Visual Hindsight Self-Imitation Learning for Interactive Navigationcited by 6
  4. 2023Learning with Chemical versus Electrical Synapses -- Does it Make a Difference?cited by 5
  5. 2024Hierarchical end-to-end autonomous navigation through few-shot waypoint detectioncited by 3
  6. 2024CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interactioncited by 3
  7. 2023One-Shot Learning of Visual Path Navigation for Autonomous Vehiclescited by 1
  8. 2023On Combining Expert Demonstrations in Imitation Learning via Optimal Transportcited by 1
  9. 2023Active Third-Person Imitation Learningcited by 1
  10. 2024Dyna-LfLH: Learning Agile Navigation in Dynamic Environments from Learned Hallucinationcited by 1
  11. 2025Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaborationcited by 1
  12. 2024Lift, Splat, Map: Lifting Foundation Masks for Label-Free Semantic Scene Completioncited by 0
  13. 2024Collision Avoidance for Multiple UAVs in Unknown Scenarios with Causal Representation Disentanglementcited by 0
  14. 2024Aligning Robot Navigation Behaviors with Human Intentions and Preferencescited by 0
  15. 2024Reliable Semantic Understanding for Real World Zero-shot Object Goal Navigationcited by 0
  16. 2026Feasibility-aware Imitation Learning from Observation with Multimodal Feedbackcited by 0
  17. 2026SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigationcited by 0
  18. 2026HumAIN: Human-Aware Implicit Social Robot Navigationcited by 0
  19. 2026VOMMI: Collecting and Leveraging Portable Demonstrations for Mobile Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Less Is More: Scalable Visual Navigation from Limited Data
  2. 2025Are Learning-Based Approaches Ready for Real-World Indoor Navigation? A Case for Imitation Learning
  3. 2018OIL: Observational Imitation Learning
  4. 2019Visual-based Autonomous Driving Deployment from a Stochastic and Uncertainty-aware Perspective
  5. 2026NavOL: Navigation Policy with Online Imitation Learning
  6. 2025OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generation
  7. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  8. 2026Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigation