# Imitation of Manipulation Skills Using Multiple Geometries

arXiv: https://arxiv.org/abs/2203.01171 · HTML: https://mlsanity.com/paper/2203.01171/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 28 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 5 of 5)

- [Supervised Fitting of Geometric Primitives to 3D Point Clouds](https://mlsanity.com/paper/1811.08988/) (2018)
- [Gaussians on Riemannian Manifolds: Applications for Robot Learning and Adaptive Control](https://mlsanity.com/paper/1909.05946/) (2019)
- [Inferring geometric constraints in human demonstrations](https://mlsanity.com/paper/1810.00140/) (2018)
- [Fit4CAD: A point cloud benchmark for fitting simple geometric primitives in CAD objects](https://mlsanity.com/paper/2105.06858/) (2021)
- [Probabilistic Iterative LQR for Short Time Horizon MPC](https://mlsanity.com/paper/2012.06349/) (2020)

## Abstract (arXiv)

Daily manipulation tasks are characterized by geometric primitives related to actions and object shapes. Such geometric descriptors are poorly represented by only using Cartesian coordinate systems. In this paper, we propose a learning approach to extract the optimal representation from a dictionary of coordinate systems to encode an observed movement/behavior. This is achieved by using an extension of Gaussian distributions on Riemannian manifolds, which is used to analyse a set of user demonstrations statistically, by considering multiple geometries as candidate representations of the task. We formulate the reproduction problem as a general optimal control problem based on an iterative linear quadratic regulator (iLQR), where the Gaussian distribution in the extracted coordinate systems are used to define the cost function. We apply our approach to object grasping and box opening tasks in simulation and on a 7-axis Franka Emika robot. The results show that the robot can exploit several geometries to execute the manipulation task and generalize it to new situations, by maintaining the invariant characteristics of the task in the coordinate system(s) of interest.