# Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objects

arXiv: https://arxiv.org/abs/2103.00655 · HTML: https://mlsanity.com/paper/2103.00655/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2021.

## Built on by (most cited first, 9 of 9)

- [AcTExplore: Active Tactile Exploration of Unknown Objects](https://mlsanity.com/paper/2310.08745/) (2023)
- [Robust Pivoting: Exploiting Frictional Stability Using Bilevel Optimization](https://mlsanity.com/paper/2203.11412/) (2022)
- [Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization](https://mlsanity.com/paper/2303.08965/) (2023)
- [Tactile MNIST: Benchmarking Active Tactile Perception](https://mlsanity.com/paper/2506.06361/) (2025)
- [Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch](https://mlsanity.com/paper/2212.08604/) (2022)
- [Grasp Transfer based on Self-Aligning Implicit Representations of Local Surfaces](https://mlsanity.com/paper/2308.07807/) (2023)
- [Bayesian optimization for robust robotic grasping using a sensorized compliant hand](https://mlsanity.com/paper/2410.18237/) (2024)
- [Apple: Toward General Active Perception via Reinforcement Learning](https://mlsanity.com/paper/2505.06182/) (2025)
- [Monocular Reconstruction of Neural Tactile Fields](https://mlsanity.com/paper/2602.12508/) (2026)

## Abstract (arXiv)

This paper addresses the problem of simultaneously exploring an unknown object to model its shape, using tactile sensors on robotic fingers, while also improving finger placement to optimise grasp stability. In many situations, a robot will have only a partial camera view of the near side of an observed object, for which the far side remains occluded. We show how an initial grasp attempt, based on an initial guess of the overall object shape, yields tactile glances of the far side of the object which enable the shape estimate and consequently the successive grasps to be improved. We propose a grasp exploration approach using a probabilistic representation of shape, based on Gaussian Process Implicit Surfaces. This representation enables initial partial vision data to be augmented with additional data from successive tactile glances. This is combined with a probabilistic estimate of grasp quality to refine grasp configurations. When choosing the next set of finger placements, a bi-objective optimisation method is used to mutually maximise grasp quality and improve shape representation during successive grasp attempts. Experimental results show that the proposed approach yields stable grasp configurations more efficiently than a baseline method, while also yielding improved shape estimate of the grasped object.