# The Art of Imitation: Learning Long-Horizon Manipulation Tasks from Few Demonstrations

arXiv: https://arxiv.org/abs/2407.13432 · HTML: https://mlsanity.com/paper/2407.13432/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 19 of 19)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills](https://mlsanity.com/paper/2302.04659/) (2023)
- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [RVT: Robotic View Transformer for 3D Object Manipulation](https://mlsanity.com/paper/2306.14896/) (2023)
- [Interactive Imitation Learning in Robotics: A Survey](https://mlsanity.com/paper/2211.00600/) (2022)
- [Kernelized Movement Primitives](https://mlsanity.com/paper/1708.08638/) (2017)
- [DITTO: Demonstration Imitation by Trajectory Transformation](https://mlsanity.com/paper/2403.15203/) (2024)
- [Correct Me if I am Wrong: Interactive Learning for Robotic Manipulation](https://mlsanity.com/paper/2110.03316/) (2021)
- [A New Approach to Time-Optimal Path Parameterization based on Reachability Analysis](https://mlsanity.com/paper/1707.07239/) (2017)
- [CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimation](https://mlsanity.com/paper/2312.08240/) (2023)
- [Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models](https://mlsanity.com/paper/2111.13129/) (2021)
- [The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation](https://mlsanity.com/paper/2305.04718/) (2023)
- [Task Generalization with Stability Guarantees via Elastic Dynamical System Motion Policies](https://mlsanity.com/paper/2309.01884/) (2023)
- [Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks](https://mlsanity.com/paper/2008.10471/) (2020)
- [Wasserstein Gradient Flows for Optimizing Gaussian Mixture Policies](https://mlsanity.com/paper/2305.10411/) (2023)

## Built on by (most cited first, 11 of 11)

- [ABPolicy: Asynchronous B-Spline Flow Policy for Real-Time and Smooth Robotic Manipulation](https://mlsanity.com/paper/2602.23901/) (2026)
- [Whole-Body Teleoperation for Mobile Manipulation at Zero Added Cost](https://mlsanity.com/paper/2409.15095/) (2024)
- [Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models](https://mlsanity.com/paper/2602.12734/) (2026)
- [The Unreasonable Effectiveness of Discrete-Time Gaussian Process Mixtures for Robot Policy Learning](https://mlsanity.com/paper/2505.03296/) (2025)
- [Learning Long-Horizon Robot Manipulation Skills via Privileged Action](https://mlsanity.com/paper/2502.15442/) (2025)
- [MSG: Multi-Stream Generative Policies for Sample-Efficient Robotic Manipulation](https://mlsanity.com/paper/2509.24956/) (2025)
- [AnchorD: Metric Grounding of Monocular Depth Using Factor Graphs](https://mlsanity.com/paper/2605.02667/) (2026)
- [PhaseLoRA: Control-Regime-Conditioned Low-Rank Adaptation for Continuous-Action Vision-Language-Action Policies](https://mlsanity.com/paper/2608.15285/) (2026)
- [Wavelet Policy: Imitation Learning in the Scale Domain with World Prior Memory](https://mlsanity.com/paper/2504.04991/) (2025)
- [One Hand Watches The Other: Dynamic Multi-Agent Cooperation for Sample-Efficient Bimanual Manipulation in Dynamic Environments](https://mlsanity.com/paper/2607.22119/) (2026)
- [Dream to Remember: Continual Steering of Generalist Robot Policies with World Models](https://mlsanity.com/paper/2610.13816/) (2026)

## Abstract (arXiv)

Task Parametrized Gaussian Mixture Models (TP-GMM) are a sample-efficient method for learning object-centric robot manipulation tasks. However, there are several open challenges to applying TP-GMMs in the wild. In this work, we tackle three crucial challenges synergistically. First, end-effector velocities are non-Euclidean and thus hard to model using standard GMMs. We thus propose to factorize the robot's end-effector velocity into its direction and magnitude, and model them using Riemannian GMMs. Second, we leverage the factorized velocities to segment and sequence skills from complex demonstration trajectories. Through the segmentation, we further align skill trajectories and hence leverage time as a powerful inductive bias. Third, we present a method to automatically detect relevant task parameters per skill from visual observations. Our approach enables learning complex manipulation tasks from just five demonstrations while using only RGB-D observations. Extensive experimental evaluations on RLBench demonstrate that our approach achieves state-of-the-art performance with 20-fold improved sample efficiency. Our policies generalize across different environments, object instances, and object positions, while the learned skills are reusable.