# Few-Shot Learning of Force-Based Motions From Demonstration Through Pre-training of Haptic Representation

arXiv: https://arxiv.org/abs/2309.04640 · HTML: https://mlsanity.com/paper/2309.04640/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 8 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Real-World Robot Learning with Masked Visual Pre-training](https://mlsanity.com/paper/2210.03109/) (2022)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [The Surprising Effectiveness of Representation Learning for Visual Imitation](https://mlsanity.com/paper/2112.01511/) (2021)
- [Self-Supervised Correspondence in Visuomotor Policy Learning](https://mlsanity.com/paper/1909.06933/) (2019)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [Task-Embedded Control Networks for Few-Shot Imitation Learning](https://mlsanity.com/paper/1810.03237/) (2018)
- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation](https://mlsanity.com/paper/2008.07682/) (2020)
- [Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation](https://mlsanity.com/paper/1804.08768/) (2018)
- [How to select and use tools? : Active Perception of Target Objects Using Multimodal Deep Learning](https://mlsanity.com/paper/2106.02445/) (2021)
- [Learning Visuo-Haptic Skewering Strategies for Robot-Assisted Feeding](https://mlsanity.com/paper/2211.14648/) (2022)
- [ROS-PyBullet Interface: A Framework for Reliable Contact Simulation and Human-Robot Interaction](https://mlsanity.com/paper/2210.06887/) (2022)
- [State Representation Learning from Demonstration](https://mlsanity.com/paper/1910.01738/) (2019)
- [On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation](https://mlsanity.com/paper/1710.08557/) (2017)
- [To Stir or Not to Stir: Online Estimation of Liquid Properties for Pouring Actions](https://mlsanity.com/paper/1904.02431/) (2019)

## Built on by (most cited first, 5 of 5)

- [DETACH: Cross-domain Learning for Long-Horizon Tasks via Mixture of Disentangled Experts](https://mlsanity.com/paper/2508.07842/) (2025)
- [Adaptive Wiping: Adaptive contact-rich manipulation through few-shot imitation learning with Force-Torque feedback and pre-trained object representations](https://mlsanity.com/paper/2505.06451/) (2025)
- [Online Estimation and Manipulation of Articulated Objects](https://mlsanity.com/paper/2601.01438/) (2026)
- [Semi-Supervised Neural Processes for Articulated Object Interactions](https://mlsanity.com/paper/2412.00145/) (2024)
- [Few-Shot Adaptation to Non-Stationary Environments via Latent Trend Embedding for Robotics](https://mlsanity.com/paper/2603.10373/) (2026)

## Abstract (arXiv)

In many contact-rich tasks, force sensing plays an essential role in adapting the motion to the physical properties of the manipulated object. To enable robots to capture the underlying distribution of object properties necessary for generalising learnt manipulation tasks to unseen objects, existing Learning from Demonstration (LfD) approaches require a large number of costly human demonstrations. Our proposed semi-supervised LfD approach decouples the learnt model into an haptic representation encoder and a motion generation decoder. This enables us to pre-train the first using large amount of unsupervised data, easily accessible, while using few-shot LfD to train the second, leveraging the benefits of learning skills from humans. We validate the approach on the wiping task using sponges with different stiffness and surface friction. Our results demonstrate that pre-training significantly improves the ability of the LfD model to recognise physical properties and generate desired wiping motions for unseen sponges, outperforming the LfD method without pre-training. We validate the motion generated by our semi-supervised LfD model on the physical robot hardware using the KUKA iiwa robot arm. We also validate that the haptic representation encoder, pre-trained in simulation, captures the properties of real objects, explaining its contribution to improving the generalisation of the downstream task.