# Adaptive Whole-body Robotic Tool-use Learning on Low-rigidity Plastic-made Humanoids Using Vision and Tactile Sensors

arXiv: https://arxiv.org/abs/2405.04826 · HTML: https://mlsanity.com/paper/2405.04826/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2024.

## Built on by (most cited first, 2 of 2)

- [RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills](https://mlsanity.com/paper/2506.14763/) (2025)
- [GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning](https://mlsanity.com/paper/2409.06427/) (2024)

## Abstract (arXiv)

Various robots have been developed so far; however, we face challenges in modeling the low-rigidity bodies of some robots. In particular, the deflection of the body changes during tool-use due to object grasping, resulting in significant shifts in the tool-tip position and the body's center of gravity. Moreover, this deflection varies depending on the weight and length of the tool, making these models exceptionally complex. However, there is currently no control or learning method that takes all of these effects into account. In this study, we propose a method for constructing a neural network that describes the mutual relationship among joint angle, visual information, and tactile information from the feet. We aim to train this network using the actual robot data and utilize it for tool-tip control. Additionally, we employ Parametric Bias to capture changes in this mutual relationship caused by variations in the weight and length of tools, enabling us to understand the characteristics of the grasped tool from the current sensor information. We apply this approach to the whole-body tool-use on KXR, a low-rigidity plastic-made humanoid robot, to validate its effectiveness.