# Learning from demonstration using products of experts: applications to manipulation and task prioritization

arXiv: https://arxiv.org/abs/2010.03505 · HTML: https://mlsanity.com/paper/2010.03505/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 12 of its 70 references that are in the map.
- First posted on arXiv 7 Oct 2020.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Learning Sampling Distributions for Robot Motion Planning](https://mlsanity.com/paper/1709.05448/) (2017)
- [Variational Boosting: Iteratively Refining Posterior Approximations](https://mlsanity.com/paper/1611.06585/) (2016)
- [Deep Generative Stochastic Networks Trainable by Backprop](https://mlsanity.com/paper/1306.1091/) (2013)
- [Boosting Variational Inference](https://mlsanity.com/paper/1611.05559/) (2016)
- [Ergodic Coverage In Constrained Environments Using Stochastic Trajectory Optimization](https://mlsanity.com/paper/1707.04294/) (2017)

## Built on by (most cited first, 4 of 4)

- [Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity](https://mlsanity.com/paper/2306.02652/) (2023)
- [Hierarchical Policy Blending as Inference for Reactive Robot Control](https://mlsanity.com/paper/2210.07890/) (2022)
- [Tensor Train for Global Optimization Problems in Robotics](https://mlsanity.com/paper/2206.05077/) (2022)
- [A context-adaptive policy framework for robust and reactive robotic manipulation via uncertainty-aware imitation learning](https://mlsanity.com/paper/2410.24035/) (2024)

## Abstract (arXiv)

Probability distributions are key components of many learning from demonstration (LfD) approaches. While the configuration of a manipulator is defined by its joint angles, poses are often best explained within several task spaces. In many approaches, distributions within relevant task spaces are learned independently and only combined at the control level. This simplification implies several problems that are addressed in this work. We show that the fusion of models in different task spaces can be expressed as a product of experts (PoE), where the probabilities of the models are multiplied and renormalized so that it becomes a proper distribution of joint angles. Multiple experiments are presented to show that learning the different models jointly in the PoE framework significantly improves the quality of the model. The proposed approach particularly stands out when the robot has to learn competitive or hierarchical objectives. Training the model jointly usually relies on contrastive divergence, which requires costly approximations that can affect performance. We propose an alternative strategy using variational inference and mixture model approximations. In particular, we show that the proposed approach can be extended to PoE with a nullspace structure (PoENS), where the model is able to recover tasks that are masked by the resolution of higher-level objectives.