# In-Hand Manipulation of Unknown Objects with Tactile Sensing for Insertion

arXiv: https://arxiv.org/abs/2210.13403 · HTML: https://mlsanity.com/paper/2210.13403/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2022.

## Built on by (most cited first, 6 of 6)

- [Binding Touch to Everything: Learning Unified Multimodal Tactile Representations](https://mlsanity.com/paper/2401.18084/) (2024)
- [Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing](https://mlsanity.com/paper/2312.01853/) (2023)
- [Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation](https://mlsanity.com/paper/2312.13469/) (2023)
- [Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting](https://mlsanity.com/paper/2403.09875/) (2024)
- [Feel the Force: Contact-Driven Learning from Humans](https://mlsanity.com/paper/2506.01944/) (2025)
- [GelBelt: A Vision-based Tactile Sensor for Continuous Sensing of Large Surfaces](https://mlsanity.com/paper/2501.06263/) (2025)

## Abstract (arXiv)

In this paper, we present a method to manipulate unknown objects in-hand using tactile sensing without relying on a known object model. In many cases, vision-only approaches may not be feasible; for example, due to occlusion in cluttered spaces. We address this limitation by introducing a method to reorient unknown objects using tactile sensing. It incrementally builds a probabilistic estimate of the object shape and pose during task-driven manipulation. Our approach uses Bayesian optimization to balance exploration of the global object shape with efficient task completion. To demonstrate the effectiveness of our method, we apply it to a simulated Tactile-Enabled Roller Grasper, a gripper that rolls objects in hand while collecting tactile data. We evaluate our method on an insertion task with randomly generated objects and find that it reliably reorients objects while significantly reducing the exploration time.