# TANDEM3D: Active Tactile Exploration for 3D Object Recognition

arXiv: https://arxiv.org/abs/2209.08772 · HTML: https://mlsanity.com/paper/2209.08772/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 7 of 7)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Tactile SLAM: Real-time inference of shape and pose from planar pushing](https://mlsanity.com/paper/2011.07044/) (2020)
- [TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors](https://mlsanity.com/paper/2203.00798/) (2022)
- [A Sensorized Multicurved Robot Finger with Data-driven Touch Sensing via Overlapping Light Signals](https://mlsanity.com/paper/2004.00685/) (2020)
- [Touch-based Curiosity for Sparse-Reward Tasks](https://mlsanity.com/paper/2104.00442/) (2021)
- [Active End-Effector Pose Selection for Tactile Object Recognition through Monte Carlo Tree Search](https://mlsanity.com/paper/1703.00095/) (2017)

## Built on by (most cited first, 10 of 10)

- [Learning Visuotactile Skills with Two Multifingered Hands](https://mlsanity.com/paper/2404.16823/) (2024)
- [ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data](https://mlsanity.com/paper/2406.19464/) (2024)
- [AcTExplore: Active Tactile Exploration of Unknown Objects](https://mlsanity.com/paper/2310.08745/) (2023)
- [Tactile-based Object Retrieval From Granular Media](https://mlsanity.com/paper/2402.04536/) (2024)
- [CHSEL: Producing Diverse Plausible Pose Estimates from Contact and Free Space Data](https://mlsanity.com/paper/2305.08042/) (2023)
- [TopoCut: Learning Multi-Step Cutting with Spectral Rewards and Discrete Diffusion Policies](https://mlsanity.com/paper/2509.19712/) (2025)
- [PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation](https://mlsanity.com/paper/2403.15107/) (2024)
- [Learning to Double Guess: An Active Perception Approach for Estimating the Center of Mass of Arbitrary Objects](https://mlsanity.com/paper/2502.02663/) (2025)
- [Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch](https://mlsanity.com/paper/2604.09100/) (2026)
- [TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments](https://mlsanity.com/paper/2606.24712/) (2026)

## Abstract (arXiv)

Tactile recognition of 3D objects remains a challenging task. Compared to 2D shapes, the complex geometry of 3D surfaces requires richer tactile signals, more dexterous actions, and more advanced encoding techniques. In this work, we propose TANDEM3D, a method that applies a co-training framework for exploration and decision making to 3D object recognition with tactile signals. Starting with our previous work, which introduced a co-training paradigm for 2D recognition problems, we introduce a number of advances that enable us to scale up to 3D. TANDEM3D is based on a novel encoder that builds 3D object representation from contact positions and normals using PointNet++. Furthermore, by enabling 6DOF movement, TANDEM3D explores and collects discriminative touch information with high efficiency. Our method is trained entirely in simulation and validated with real-world experiments. Compared to state-of-the-art baselines, TANDEM3D achieves higher accuracy and a lower number of actions in recognizing 3D objects and is also shown to be more robust to different types and amounts of sensor noise. Video is available at https://jxu.ai/tandem3d.