# Learning to Sequence and Blend Robot Skills via Differentiable Optimization

arXiv: https://arxiv.org/abs/2206.00559 · HTML: https://mlsanity.com/paper/2206.00559/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2022.

## Built on by (most cited first, 4 of 4)

- [Hierarchical Policy Blending as Inference for Reactive Robot Control](https://mlsanity.com/paper/2210.07890/) (2022)
- [Hierarchical Policy Blending As Optimal Transport](https://mlsanity.com/paper/2212.01938/) (2022)
- [Logic Learning from Demonstrations for Multi-step Manipulation Tasks in Dynamic Environments](https://mlsanity.com/paper/2404.16138/) (2024)
- [Information Theoretic Bayesian Optimization over the Probability Simplex](https://mlsanity.com/paper/2603.09793/) (2026)

## Abstract (arXiv)

In contrast to humans and animals who naturally execute seamless motions, learning and smoothly executing sequences of actions remains a challenge in robotics. This paper introduces a novel skill-agnostic framework that learns to sequence and blend skills based on differentiable optimization. Our approach encodes sequences of previously-defined skills as quadratic programs (QP), whose parameters determine the relative importance of skills along the task. Seamless skill sequences are then learned from demonstrations by exploiting differentiable optimization layers and a tailored loss formulated from the QP optimality conditions. Via the use of differentiable optimization, our work offers novel perspectives on multitask control. We validate our approach in a pick-and-place scenario with planar robots, a pouring experiment with a real humanoid robot, and a bimanual sweeping task with a human model.