# Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation

arXiv: https://arxiv.org/abs/1909.12971 · HTML: https://mlsanity.com/paper/1909.12971/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 27 Sep 2019.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Time-Contrastive Networks: Self-Supervised Learning from Video](https://mlsanity.com/paper/1704.06888/) (2017)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Searching for Activation Functions](https://mlsanity.com/paper/1710.05941/) (2017)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [Zero-Shot Visual Imitation](https://mlsanity.com/paper/1804.08606/) (2018)
- [Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks](https://mlsanity.com/paper/1903.03878/) (2019)
- [Third-Person Imitation Learning](https://mlsanity.com/paper/1703.01703/) (2017)
- [Deep Imitative Models for Flexible Inference, Planning, and Control](https://mlsanity.com/paper/1810.06544/) (2018)
- [Learning Navigation Behaviors End-to-End with AutoRL](https://mlsanity.com/paper/1809.10124/) (2018)
- [From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following](https://mlsanity.com/paper/1902.07742/) (2019)
- [Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders](https://mlsanity.com/paper/1804.10469/) (2018)
- [Visual Semantic Planning using Deep Successor Representations](https://mlsanity.com/paper/1705.08080/) (2017)
- [Long-Range Indoor Navigation with PRM-RL](https://mlsanity.com/paper/1902.09458/) (2019)
- [Universal Planning Networks](https://mlsanity.com/paper/1804.00645/) (2018)
- [Learning Navigation Subroutines from Egocentric Videos](https://mlsanity.com/paper/1905.12612/) (2019)

## Built on by (most cited first, 13 of 13)

- [No RL, No Simulation: Learning to Navigate without Navigating](https://mlsanity.com/paper/2110.09470/) (2021)
- [AffordDP: Generalizable Diffusion Policy with Transferable Affordance](https://mlsanity.com/paper/2412.03142/) (2024)
- [VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation](https://mlsanity.com/paper/2105.09371/) (2021)
- [Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning](https://mlsanity.com/paper/2403.03848/) (2024)
- [Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments](https://mlsanity.com/paper/2209.09233/) (2022)
- [One-Shot Learning from a Demonstration with Hierarchical Latent Language](https://mlsanity.com/paper/2203.04806/) (2022)
- [Instance-Aware Predictive Navigation in Multi-Agent Environments](https://mlsanity.com/paper/2101.05893/) (2021)
- [Data-Efficient Learning from Human Interventions for Mobile Robots](https://mlsanity.com/paper/2503.04969/) (2025)
- [Unlocking Pixels for Reinforcement Learning via Implicit Attention](https://mlsanity.com/paper/2102.04353/) (2021)
- [CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interaction](https://mlsanity.com/paper/2410.01273/) (2024)
- [Zero-Shot Offline Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2410.08751/) (2024)
- [McARL:Morphology-Control-Aware Reinforcement Learning for Generalizable Quadrupedal Locomotion](https://mlsanity.com/paper/2505.18418/) (2025)
- [From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation](https://mlsanity.com/paper/2606.12603/) (2026)

## Abstract (arXiv)

Imitation learning is a popular approach for training visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for a long time. Here, we propose a zero-shot imitation learning approach for training a visual navigation policy on legged robots from human (third-person perspective) demonstrations, enabling high-quality navigation and cost-effective data collection. However, imitation learning from third-person demonstrations raises unique challenges. First, these demonstrations are captured from different camera perspectives, which we address via a feature disentanglement network (FDN) that extracts perspective-invariant state features. Second, as transition dynamics vary across systems, we label missing actions by either building an inverse model of the robot's dynamics in the feature space and applying it to the human demonstrations or developing a Graphic User Interface(GUI) to label human demonstrations. To train a navigation policy we use a model-based imitation learning approach with FDN and labeled human demonstrations. We show that our framework can learn an effective policy for a legged robot, Laikago, from human demonstrations in both simulated and real-world environments. Our approach is zero-shot as the robot never navigates the same paths during training as those at testing time. We justify our framework by performing a comparative study.