# Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation

arXiv: https://arxiv.org/abs/2210.11779 · HTML: https://mlsanity.com/paper/2210.11779/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [InfoVAE: Information Maximizing Variational Autoencoders](https://mlsanity.com/paper/1706.02262/) (2017)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [Motion Planning Networks](https://mlsanity.com/paper/1806.05767/) (2018)
- [Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference](https://mlsanity.com/paper/1707.07383/) (2017)
- [Riemannian Motion Policies](https://mlsanity.com/paper/1801.02854/) (2018)
- [Robot Motion Planning in Learned Latent Spaces](https://mlsanity.com/paper/1807.10366/) (2018)
- [Robust Locally-Linear Controllable Embedding](https://mlsanity.com/paper/1710.05373/) (2017)
- [Fast Marching Tree: a Fast Marching Sampling-Based Method for Optimal Motion Planning in Many Dimensions](https://mlsanity.com/paper/1306.3532/) (2013)
- [Batch Informed Trees (BIT*): Informed Asymptotically Optimal Anytime Search](https://mlsanity.com/paper/1707.01888/) (2017)
- [Neural Manipulation Planning on Constraint Manifolds](https://mlsanity.com/paper/2008.03787/) (2020)
- [First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion](https://mlsanity.com/paper/2007.01520/) (2020)

## Built on by (most cited first, 8 of 8)

- [Neural Contractive Dynamical Systems](https://mlsanity.com/paper/2401.09352/) (2024)
- [Leveraging Scene Embeddings for Gradient-Based Motion Planning in Latent Space](https://mlsanity.com/paper/2303.03364/) (2023)
- [Fine-Tuning Generative Models as an Inference Method for Robotic Tasks](https://mlsanity.com/paper/2310.12862/) (2023)
- [Approximate Shielding of Atari Agents for Safe Exploration](https://mlsanity.com/paper/2304.11104/) (2023)
- [Extended Neural Contractive Dynamical Systems: On Multiple Tasks and Riemannian Safety Regions](https://mlsanity.com/paper/2411.11405/) (2024)
- [Toward Generalist Neural Motion Planners for Robotic Manipulators: Challenges and Opportunities](https://mlsanity.com/paper/2603.24318/) (2026)
- [Improving Trajectory Stitching with Flow Models](https://mlsanity.com/paper/2505.07802/) (2025)
- [A Cross-Environment and Cross-Embodiment Path Planning Framework via a Conditional Diffusion Model](https://mlsanity.com/paper/2510.19128/) (2025)

## Abstract (arXiv)

We present a novel approach to path planning for robotic manipulators, in which paths are produced via iterative optimisation in the latent space of a generative model of robot poses. Constraints are incorporated through the use of constraint satisfaction classifiers operating on the same space. Optimisation leverages gradients through our learned models that provide a simple way to combine goal reaching objectives with constraint satisfaction, even in the presence of otherwise non-differentiable constraints. Our models are trained in a task-agnostic manner on randomly sampled robot poses. In baseline comparisons against a number of widely used planners, we achieve commensurate performance in terms of task success, planning time and path length, performing successful path planning with obstacle avoidance on a real 7-DoF robot arm.