# A Geometric Perspective on Visual Imitation Learning

arXiv: https://arxiv.org/abs/2003.02768 · HTML: https://mlsanity.com/paper/2003.02768/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 5 Mar 2020.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 15 of 15)

- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation](https://mlsanity.com/paper/1806.08756/) (2018)
- [Zero-Shot Visual Imitation](https://mlsanity.com/paper/1804.08606/) (2018)
- [Self-Supervised Correspondence in Visuomotor Policy Learning](https://mlsanity.com/paper/1909.06933/) (2019)
- [Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://mlsanity.com/paper/1911.09676/) (2019)
- [KETO: Learning Keypoint Representations for Tool Manipulation](https://mlsanity.com/paper/1910.11977/) (2019)
- [Graph-Structured Visual Imitation](https://mlsanity.com/paper/1907.05518/) (2019)
- [A survey of robot learning from demonstrations for Human-Robot Collaboration](https://mlsanity.com/paper/1710.08789/) (2017)
- [Visual Geometric Skill Inference by Watching Human Demonstration](https://mlsanity.com/paper/1911.04418/) (2019)
- [Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach](https://mlsanity.com/paper/1810.00159/) (2018)
- [Evaluation of state representation methods in robot hand-eye coordination learning from demonstration](https://mlsanity.com/paper/1903.00634/) (2019)

## Built on by (most cited first, 10 of 10)

- [Learning Generalizable Dexterous Manipulation from Human Grasp Affordance](https://mlsanity.com/paper/2204.02320/) (2022)
- [K-VIL: Keypoints-based Visual Imitation Learning](https://mlsanity.com/paper/2209.03277/) (2022)
- [Imitation Learning for Variable Speed Contact Motion for Operation up to Control Bandwidth](https://mlsanity.com/paper/2102.10283/) (2021)
- [Bi-KVIL: Keypoints-based Visual Imitation Learning of Bimanual Manipulation Tasks](https://mlsanity.com/paper/2403.03270/) (2024)
- [VIEW: Visual Imitation Learning with Waypoints](https://mlsanity.com/paper/2404.17906/) (2024)
- [Dynamic Rank Adjustment in Diffusion Policies for Efficient and Flexible Training](https://mlsanity.com/paper/2502.03822/) (2025)
- [Generalizable task representation learning from human demonstration videos: a geometric approach](https://mlsanity.com/paper/2202.13604/) (2022)
- [One-shot Visual Imitation via Attributed Waypoints and Demonstration Augmentation](https://mlsanity.com/paper/2302.04856/) (2023)
- [FRMD: Fast Robot Motion Diffusion with Consistency-Distilled Movement Primitives for Smooth Action Generation](https://mlsanity.com/paper/2503.02048/) (2025)
- [Bilateral Control-Based Imitation Learning for Velocity-Controlled Robot](https://mlsanity.com/paper/2103.04004/) (2021)

## Abstract (arXiv)

We consider the problem of visual imitation learning without human supervision (e.g. kinesthetic teaching or teleoperation), nor access to an interactive reinforcement learning (RL) training environment. We present a geometric perspective to derive solutions to this problem. Specifically, we propose VGS-IL (Visual Geometric Skill Imitation Learning), an end-to-end geometry-parameterized task concept inference method, to infer globally consistent geometric feature association rules from human demonstration video frames. We show that, instead of learning actions from image pixels, learning a geometry-parameterized task concept provides an explainable and invariant representation across demonstrator to imitator under various environmental settings. Moreover, such a task concept representation provides a direct link with geometric vision based controllers (e.g. visual servoing), allowing for efficient mapping of high-level task concepts to low-level robot actions.