# A General Mobile Manipulator Automation Framework for Flexible Manufacturing in Hostile Industrial Environments

arXiv: https://arxiv.org/abs/2302.04486 · HTML: https://mlsanity.com/paper/2302.04486/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 62 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 11 of 11)

- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration](https://mlsanity.com/paper/1605.03344/) (2016)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration](https://mlsanity.com/paper/2011.12149/) (2020)
- [A comprehensive survey on point cloud registration](https://mlsanity.com/paper/2103.02690/) (2021)
- [Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots](https://mlsanity.com/paper/2203.01389/) (2022)
- [LSG-CPD: Coherent Point Drift with Local Surface Geometry for Point Cloud Registration](https://mlsanity.com/paper/2103.15039/) (2021)
- [Learning Actions from Human Demonstration Video for Robotic Manipulation](https://mlsanity.com/paper/1909.04312/) (2019)
- [Sdf-GAN: Semi-supervised Depth Fusion with Multi-scale Adversarial Networks](https://mlsanity.com/paper/1803.06657/) (2018)
- [DUGMA: Dynamic Uncertainty-Based Gaussian Mixture Alignment](https://mlsanity.com/paper/1803.07426/) (2018)
- [UDFNet: Unsupervised Disparity Fusion with Adversarial Networks](https://mlsanity.com/paper/1904.10044/) (2019)

## Abstract (arXiv)

To enable a mobile manipulator to perform human tasks from a single teaching demonstration is vital to flexible manufacturing. We call our proposed method MMPA (Mobile Manipulator Process Automation with One-shot Teaching). Currently, there is no effective and robust MMPA framework which is not influenced by harsh industrial environments and the mobile base's parking precision. The proposed MMPA framework consists of two stages: collecting data (mobile base's location, environment information, end-effector's path) in the teaching stage for robot learning; letting the end-effector repeat the nearly same path as the reference path in the world frame to reproduce the work in the automation stage. More specifically, in the automation stage, the robot navigates to the specified location without the need of a precise parking. Then, based on colored point cloud registration, the proposed IPE (Iterative Pose Estimation by Eye & Hand) algorithm could estimate the accurate 6D relative parking pose of the robot arm base without the need of any marker. Finally, the robot could learn the error compensation from the parking pose's bias to modify the end-effector's path to make it repeat a nearly same path in the world coordinate system as recorded in the teaching stage. Hundreds of trials have been conducted with a real mobile manipulator to show the superior robustness of the system and the accuracy of the process automation regardless of the harsh industrial conditions and parking precision. For the released code, please contact marketing@amigaga.com