# Reactive Motion Generation on Learned Riemannian Manifolds

arXiv: https://arxiv.org/abs/2203.07761 · HTML: https://mlsanity.com/paper/2203.07761/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 64 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [Neural Dynamic Policies for End-to-End Sensorimotor Learning](https://mlsanity.com/paper/2012.02788/) (2020)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [Riemannian Motion Policies](https://mlsanity.com/paper/1801.02854/) (2018)
- [Geometrically Enriched Latent Spaces](https://mlsanity.com/paper/2008.00565/) (2020)
- [Kernelized Movement Primitives](https://mlsanity.com/paper/1708.08638/) (2017)
- [Variational Autoencoders with Riemannian Brownian Motion Priors](https://mlsanity.com/paper/2002.05227/) (2020)
- [Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems](https://mlsanity.com/paper/2005.13143/) (2020)
- [Fast and Robust Shortest Paths on Manifolds Learned from Data](https://mlsanity.com/paper/1901.07229/) (2019)
- [Bayesian Optimization Meets Riemannian Manifolds in Robot Learning](https://mlsanity.com/paper/1910.04998/) (2019)
- [Learning Riemannian Manifolds for Geodesic Motion Skills](https://mlsanity.com/paper/2106.04315/) (2021)
- [Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning](https://mlsanity.com/paper/2105.04962/) (2021)
- [Directional Statistics in Machine Learning: a Brief Review](https://mlsanity.com/paper/1605.00316/) (2016)
- [Fast Approximate Geodesics for Deep Generative Models](https://mlsanity.com/paper/1812.08284/) (2018)
- [Learning Vision-based Reactive Policies for Obstacle Avoidance](https://mlsanity.com/paper/2010.16298/) (2020)
- [Orientation Probabilistic Movement Primitives on Riemannian Manifolds](https://mlsanity.com/paper/2110.15036/) (2021)
- [Expected path length on random manifolds](https://mlsanity.com/paper/1908.07377/) (2019)
- [Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks](https://mlsanity.com/paper/2008.10471/) (2020)
- [Goal-Conditioned Variational Autoencoder Trajectory Primitives with Continuous and Discrete Latent Codes](https://mlsanity.com/paper/1912.04063/) (2019)

## Built on by (most cited first, 6 of 6)

- [Learning Riemannian Stable Dynamical Systems via Diffeomorphisms](https://mlsanity.com/paper/2211.03169/) (2022)
- [Learning Neural Contracting Dynamics: Extended Linearization and Global Guarantees](https://mlsanity.com/paper/2402.08090/) (2024)
- [Adversarial robustness of VAEs through the lens of local geometry](https://mlsanity.com/paper/2208.03923/) (2022)
- [Geometric Contact Flows: Contactomorphisms for Dynamics and Control](https://mlsanity.com/paper/2506.17868/) (2025)
- [Extended Neural Contractive Dynamical Systems: On Multiple Tasks and Riemannian Safety Regions](https://mlsanity.com/paper/2411.11405/) (2024)
- [The Field Knows: Cross-Dimensional Geometry from Navigation to Black Holes](https://mlsanity.com/paper/2608.07566/) (2026)

## Abstract (arXiv)

In recent decades, advancements in motion learning have enabled robots to acquire new skills and adapt to unseen conditions in both structured and unstructured environments. In practice, motion learning methods capture relevant patterns and adjust them to new conditions such as dynamic obstacle avoidance or variable targets. In this paper, we investigate the robot motion learning paradigm from a Riemannian manifold perspective. We argue that Riemannian manifolds may be learned via human demonstrations in which geodesics are natural motion skills. The geodesics are generated using a learned Riemannian metric produced by our novel variational autoencoder (VAE), which is especially intended to recover full-pose end-effector states and joint space configurations. In addition, we propose a technique for facilitating on-the-fly end-effector/multiple-limb obstacle avoidance by reshaping the learned manifold using an obstacle-aware ambient metric. The motion generated using these geodesics may naturally result in multiple-solution tasks that have not been explicitly demonstrated previously. We extensively tested our approach in task space and joint space scenarios using a 7-DoF robotic manipulator. We demonstrate that our method is capable of learning and generating motion skills based on complicated motion patterns demonstrated by a human operator. Additionally, we assess several obstacle avoidance strategies and generate trajectories in multiple-mode settings.