# Learning Forceful Manipulation Skills from Multi-modal Human Demonstrations

arXiv: https://arxiv.org/abs/2109.04222 · HTML: https://mlsanity.com/paper/2109.04222/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 9 Sep 2021.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Built on by (most cited first, 7 of 7)

- [Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models](https://mlsanity.com/paper/2308.01557/) (2023)
- [OpenVR: Teleoperation for Manipulation](https://mlsanity.com/paper/2305.09765/) (2023)
- [Curriculum-Based Imitation of Versatile Skills](https://mlsanity.com/paper/2304.05171/) (2023)
- [Geometric Contact Flows: Contactomorphisms for Dynamics and Control](https://mlsanity.com/paper/2506.17868/) (2025)
- [Action ControlNet: A Lightweight Delay-Aware Adapter for Smooth Asynchronous Control in Vision-Language-Action Models](https://mlsanity.com/paper/2606.25985/) (2026)
- [VIDP: Variable Impedance Diffusion Policy for Compliant Robot Manipulation from Diverse Demonstrations](https://mlsanity.com/paper/2608.06210/) (2026)
- [One-Shot Learning from Demonstration of Contact-Rich Robotic Manipulation by Identifying Physical Interactions](https://mlsanity.com/paper/2608.24741/) (2026)

## Abstract (arXiv)

Learning from Demonstration (LfD) provides an intuitive and fast approach to program robotic manipulators. Task parameterized representations allow easy adaptation to new scenes and online observations. However, this approach has been limited to pose-only demonstrations and thus only skills with spatial and temporal features. In this work, we extend the LfD framework to address forceful manipulation skills, which are of great importance for industrial processes such as assembly. For such skills, multi-modal demonstrations including robot end-effector poses, force and torque readings, and operation scene are essential. Our objective is to reproduce such skills reliably according to the demonstrated pose and force profiles within different scenes. The proposed method combines our previous work on task-parameterized optimization and attractor-based impedance control. The learned skill model consists of (i) the attractor model that unifies the pose and force features, and (ii) the stiffness model that optimizes the stiffness for different stages of the skill. Furthermore, an online execution algorithm is proposed to adapt the skill execution to real-time observations of robot poses, measured forces, and changed scenes. We validate this method rigorously on a 7-DoF robot arm over several steps of an E-bike motor assembly process, which require different types of forceful interaction such as insertion, sliding and twisting.