# Sliding Touch-based Exploration for Modeling Unknown Object Shape with Multi-fingered Hands

arXiv: https://arxiv.org/abs/2308.00576 · HTML: https://mlsanity.com/paper/2308.00576/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2023.

## Built on by (most cited first, 6 of 6)

- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations](https://mlsanity.com/paper/2408.04380/) (2024)
- [Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation](https://mlsanity.com/paper/2312.13469/) (2023)
- [Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting](https://mlsanity.com/paper/2403.09875/) (2024)
- [Sim2Real Bilevel Adaptation for Object Surface Classification using Vision-Based Tactile Sensors](https://mlsanity.com/paper/2311.01380/) (2023)
- [Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch](https://mlsanity.com/paper/2604.09100/) (2026)

## Abstract (arXiv)

Efficient and accurate 3D object shape reconstruction contributes significantly to the success of a robot's physical interaction with its environment. Acquiring accurate shape information about unknown objects is challenging, especially in unstructured environments, e.g. the vision sensors may only be able to provide a partial view. To address this issue, tactile sensors could be employed to extract local surface information for more robust unknown object shape estimation. In this paper, we propose a novel approach for efficient unknown 3D object shape exploration and reconstruction using a multi-fingered hand equipped with tactile sensors and a depth camera only providing a partial view. We present a multi-finger sliding touch strategy for efficient shape exploration using a Bayesian Optimization approach and a single-leader-multi-follower strategy for multi-finger smooth local surface perception. We evaluate our proposed method by estimating the 3D shape of objects from the YCB and OCRTOC datasets based on simulation and real robot experiments. The proposed approach yields successful reconstruction results relying on only a few continuous sliding touches. Experimental results demonstrate that our method is able to model unknown objects in an efficient and accurate way.