# Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skills

arXiv: https://arxiv.org/abs/2010.13766 · HTML: https://mlsanity.com/paper/2010.13766/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 26 Oct 2020.

## Placed in

- Offline RL and imitation learning from data / [Real-world robot offline RL](https://mlsanity.com/offlinerl/real-world-robot-offline-rl/)

## Built on by (most cited first, 5 of 5)

- [Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies](https://mlsanity.com/paper/2112.05062/) (2021)
- [Variational Distillation of Diffusion Policies into Mixture of Experts](https://mlsanity.com/paper/2406.12538/) (2024)
- [Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts](https://mlsanity.com/paper/2403.06966/) (2024)
- [Deep Probabilistic Movement Primitives with a Bayesian Aggregator](https://mlsanity.com/paper/2307.05141/) (2023)
- [Dimensionality Reduction and Prioritized Exploration for Policy Search](https://mlsanity.com/paper/2203.04791/) (2022)

## Abstract (arXiv)

Parameterized movement primitives have been extensively used for imitation learning of robotic tasks. However, the high-dimensionality of the parameter space hinders the improvement of such primitives in the reinforcement learning (RL) setting, especially for learning with physical robots. In this paper we propose a novel view on handling the demonstrated trajectories for acquiring low-dimensional, non-linear latent dynamics, using mixtures of probabilistic principal component analyzers (MPPCA) on the movements' parameter space. Moreover, we introduce a new contextual off-policy RL algorithm, named LAtent-Movements Policy Optimization (LAMPO). LAMPO can provide gradient estimates from previous experience using self-normalized importance sampling, hence, making full use of samples collected in previous learning iterations. These advantages combined provide a complete framework for sample-efficient off-policy optimization of movement primitives for robot learning of high-dimensional manipulation skills. Our experimental results conducted both in simulation and on a real robot show that LAMPO provides sample-efficient policies against common approaches in literature.