# Object Pose Estimation through Dexterous Touch

arXiv: https://arxiv.org/abs/2509.13591 · HTML: https://mlsanity.com/paper/2509.13591/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Builds on (most cited first, 30 of 33)

- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation](https://mlsanity.com/paper/2005.14679/) (2020)
- [Rotating without Seeing: Towards In-hand Dexterity through Touch](https://mlsanity.com/paper/2303.10880/) (2023)
- [Learning Visuotactile Skills with Two Multifingered Hands](https://mlsanity.com/paper/2404.16823/) (2024)
- [3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing](https://mlsanity.com/paper/2410.24091/) (2024)
- [Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge](https://mlsanity.com/paper/1609.09475/) (2016)
- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation](https://mlsanity.com/paper/2203.04802/) (2022)
- [GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger](https://mlsanity.com/paper/1803.00628/) (2018)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [The Curious Robot: Learning Visual Representations via Physical Interactions](https://mlsanity.com/paper/1604.01360/) (2016)
- [Active 3D Shape Reconstruction from Vision and Touch](https://mlsanity.com/paper/2107.09584/) (2021)
- [ShapeMap 3-D: Efficient shape mapping through dense touch and vision](https://mlsanity.com/paper/2109.09884/) (2021)
- [Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation](https://mlsanity.com/paper/2312.13469/) (2023)
- [Lessons from Learning to Spin "Pens"](https://mlsanity.com/paper/2407.18902/) (2024)
- [Challenges for Monocular 6D Object Pose Estimation in Robotics](https://mlsanity.com/paper/2307.12172/) (2023)
- [Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering](https://mlsanity.com/paper/2012.05205/) (2020)
- [Active Extrinsic Contact Sensing: Application to General Peg-in-Hole Insertion](https://mlsanity.com/paper/2110.03555/) (2021)
- [TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing](https://mlsanity.com/paper/2311.12602/) (2023)
- [TransNet: Category-Level Transparent Object Pose Estimation](https://mlsanity.com/paper/2208.10002/) (2022)
- [TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors](https://mlsanity.com/paper/2203.00798/) (2022)
- [AcTExplore: Active Tactile Exploration of Unknown Objects](https://mlsanity.com/paper/2310.08745/) (2023)
- [Active Visuo-Haptic Object Shape Completion](https://mlsanity.com/paper/2203.09149/) (2022)
- [FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors](https://mlsanity.com/paper/2410.02048/) (2024)
- [Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensors](https://mlsanity.com/paper/2301.13667/) (2023)
- [ManiPose: A Comprehensive Benchmark for Pose-aware Object Manipulation in Robotics](https://mlsanity.com/paper/2403.13365/) (2024)
- [Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing](https://mlsanity.com/paper/2210.04026/) (2022)

## Abstract (arXiv)

Robust object pose estimation is essential for manipulation and interaction tasks in robotics, particularly in scenarios where visual data is limited or sensitive to lighting, occlusions, and appearances. Tactile sensors often offer limited and local contact information, making it challenging to reconstruct the pose from partial data. Our approach uses sensorimotor exploration to actively control a robot hand to interact with the object. We train with Reinforcement Learning (RL) to explore and collect tactile data. The collected 3D point clouds are used to iteratively refine the object's shape and pose. In our setup, one hand holds the object steady while the other performs active exploration. We show that our method can actively explore an object's surface to identify critical pose features without prior knowledge of the object's geometry. Supplementary material and more demonstrations will be provided at https://amirshahid.github.io/BimanualTactilePose .