# Learning Riemannian Manifolds for Geodesic Motion Skills

arXiv: https://arxiv.org/abs/2106.04315 · HTML: https://mlsanity.com/paper/2106.04315/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [Neural Dynamic Policies for End-to-End Sensorimotor Learning](https://mlsanity.com/paper/2012.02788/) (2020)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [Geometrically Enriched Latent Spaces](https://mlsanity.com/paper/2008.00565/) (2020)
- [Kernelized Movement Primitives](https://mlsanity.com/paper/1708.08638/) (2017)
- [Variational Autoencoders with Riemannian Brownian Motion Priors](https://mlsanity.com/paper/2002.05227/) (2020)
- [Fast and Robust Shortest Paths on Manifolds Learned from Data](https://mlsanity.com/paper/1901.07229/) (2019)
- [Bayesian Optimization Meets Riemannian Manifolds in Robot Learning](https://mlsanity.com/paper/1910.04998/) (2019)
- [Directional Statistics in Machine Learning: a Brief Review](https://mlsanity.com/paper/1605.00316/) (2016)
- [Fast Approximate Geodesics for Deep Generative Models](https://mlsanity.com/paper/1812.08284/) (2018)
- [Expected path length on random manifolds](https://mlsanity.com/paper/1908.07377/) (2019)
- [Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks](https://mlsanity.com/paper/2008.10471/) (2020)

## Built on by (most cited first, 23 of 23)

- [Pulling back information geometry](https://mlsanity.com/paper/2106.05367/) (2021)
- [Learning Riemannian Stable Dynamical Systems via Diffeomorphisms](https://mlsanity.com/paper/2211.03169/) (2022)
- [Neural Optimal Transport with Lagrangian Costs](https://mlsanity.com/paper/2406.00288/) (2024)
- [Neural Contractive Dynamical Systems](https://mlsanity.com/paper/2401.09352/) (2024)
- [Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models](https://mlsanity.com/paper/2505.18230/) (2025)
- [Reactive Motion Generation on Learned Riemannian Manifolds](https://mlsanity.com/paper/2203.07761/) (2022)
- [Decoder ensembling for learned latent geometries](https://mlsanity.com/paper/2408.07507/) (2024)
- [Riemann2: Learning Riemannian Submanifolds from Riemannian Data](https://mlsanity.com/paper/2503.05540/) (2025)
- [Mario Plays on a Manifold: Generating Functional Content in Latent Space through Differential Geometry](https://mlsanity.com/paper/2206.00106/) (2022)
- [MMP++: Motion Manifold Primitives with Parametric Curve Models](https://mlsanity.com/paper/2310.17072/) (2023)
- [Identifying latent distances with Finslerian geometry](https://mlsanity.com/paper/2212.10010/) (2022)
- [Identifying Metric Structures of Deep Latent Variable Models](https://mlsanity.com/paper/2502.13757/) (2025)
- [Unified Control Framework for Real-Time Interception and Obstacle Avoidance of Fast-Moving Objects with Diffusion Variational Autoencoder](https://mlsanity.com/paper/2209.13628/) (2022)
- [Learning Dynamical Systems Encoding Non-Linearity within Space Curvature](https://mlsanity.com/paper/2403.11948/) (2024)
- [Motion Manifold Flow Primitives for Task-Conditioned Trajectory Generation under Complex Task-Motion Dependencies](https://mlsanity.com/paper/2407.19681/) (2024)
- [SERA: Safe and Efficient Reactive Obstacle Avoidance for Collaborative Robotic Planning in Unstructured Environments](https://mlsanity.com/paper/2203.13821/) (2022)
- [A Structured Prediction Approach for Robot Imitation Learning](https://mlsanity.com/paper/2309.14829/) (2023)
- [On Probabilistic Pullback Metrics for Latent Hyperbolic Manifolds](https://mlsanity.com/paper/2410.20850/) (2024)
- [Extended Neural Contractive Dynamical Systems: On Multiple Tasks and Riemannian Safety Regions](https://mlsanity.com/paper/2411.11405/) (2024)
- [Deep Metric Tensor Regularized Policy Gradient](https://mlsanity.com/paper/2305.11017/) (2023)
- [Solving Functional Optimization with Deep Networks and Variational Principles](https://mlsanity.com/paper/2410.06277/) (2024)
- [Learning Topology-Driven Multi-Subspace Fusion for Grassmannian Deep Network](https://mlsanity.com/paper/2511.08628/) (2025)
- [Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods](https://mlsanity.com/paper/2609.31107/) (2026)

## Abstract (arXiv)

For robots to work alongside humans and perform in unstructured environments, they must learn new motion skills and adapt them to unseen situations on the fly. This demands learning models that capture relevant motion patterns, while offering enough flexibility to adapt the encoded skills to new requirements, such as dynamic obstacle avoidance. We introduce a Riemannian manifold perspective on this problem, and propose to learn a Riemannian manifold from human demonstrations on which geodesics are natural motion skills. We realize this with a variational autoencoder (VAE) over the space of position and orientations of the robot end-effector. Geodesic motion skills let a robot plan movements from and to arbitrary points on the data manifold. They also provide a straightforward method to avoid obstacles by redefining the ambient metric in an online fashion. Moreover, geodesics naturally exploit the manifold resulting from multiple--mode tasks to design motions that were not explicitly demonstrated previously. We test our learning framework using a 7-DoF robotic manipulator, where the robot satisfactorily learns and reproduces realistic skills featuring elaborated motion patterns, avoids previously unseen obstacles, and generates novel movements in multiple-mode settings.