# Conditional Neural Expert Processes for Learning Movement Primitives from Demonstration

arXiv: https://arxiv.org/abs/2402.08424 · HTML: https://mlsanity.com/paper/2402.08424/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Builds on (most cited first, 10 of 10)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms](https://mlsanity.com/paper/1907.03146/) (2019)
- [Gaussian Processes for Data-Efficient Learning in Robotics and Control](https://mlsanity.com/paper/1502.02860/) (2015)
- [Dynamic Movement Primitives in Robotics: A Tutorial Survey](https://mlsanity.com/paper/2102.03861/) (2021)
- [Bayesian Gaussian mixture model for robotic policy imitation](https://mlsanity.com/paper/1904.10716/) (2019)
- [Gaussian-Process-based Robot Learning from Demonstration](https://mlsanity.com/paper/2002.09979/) (2020)
- [Learning Social Navigation from Demonstrations with Conditional Neural Processes](https://mlsanity.com/paper/2210.03582/) (2022)
- [Mixture Models for the Analysis, Edition, and Synthesis of Continuous Time Series](https://mlsanity.com/paper/2104.10731/) (2021)
- [Meta-World Conditional Neural Processes](https://mlsanity.com/paper/2302.10320/) (2023)

## Built on by (most cited first, 3 of 3)

- [Affordance-based Robot Manipulation with Flow Matching](https://mlsanity.com/paper/2409.01083/) (2024)
- [RAMPA: Robotic Augmented Reality for Machine Programming by DemonstrAtion](https://mlsanity.com/paper/2410.13412/) (2024)
- [A context-adaptive policy framework for robust and reactive robotic manipulation via uncertainty-aware imitation learning](https://mlsanity.com/paper/2410.24035/) (2024)

## Abstract (arXiv)

Learning from Demonstration (LfD) is a widely used technique for skill acquisition in robotics. However, demonstrations of the same skill may exhibit significant variances, or learning systems may attempt to acquire different means of the same skill simultaneously, making it challenging to encode these motions into movement primitives. To address these challenges, we propose an LfD framework, namely the Conditional Neural Expert Processes (CNEP), that learns to assign demonstrations from different modes to distinct expert networks utilizing the inherent information within the latent space to match experts with the encoded representations. CNEP does not require supervision on which mode the trajectories belong to. We compare the performance of CNEP against widely used and powerful LfD methods such as Gaussian Mixture Models, Probabilistic Movement Primitives, and Stable Movement Primitives and show that our method outperforms these baselines on multimodal trajectory datasets. The results reveal enhanced modeling performance for movement primitives, leading to the synthesis of trajectories that more accurately reflect those demonstrated by experts, particularly when the skill demonstrations include intersection points from various trajectories. We evaluated the CNEP model on two real-robot tasks, namely obstacle avoidance and pick-and-place tasks, that require the robot to learn multi-modal motion trajectories and execute the correct primitives given target environment conditions. We also showed that our system is capable of on-the-fly adaptation to environmental changes via an online conditioning mechanism. Lastly, we believe that CNEP offers improved explainability and interpretability by autonomously finding discrete behavior primitives and providing probability values about its expert selection decisions.