# Active Visuo-Tactile Interactive Robotic Perception for Accurate Object Pose Estimation in Dense Clutter

arXiv: https://arxiv.org/abs/2202.02207 · HTML: https://mlsanity.com/paper/2202.02207/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2022.

## Built on by (most cited first, 7 of 7)

- [FastMAC: Stochastic Spectral Sampling of Correspondence Graph](https://mlsanity.com/paper/2403.08770/) (2024)
- [Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensors](https://mlsanity.com/paper/2301.13667/) (2023)
- [GelSLAM: A Real-time, High-Fidelity, and Robust 3D Tactile SLAM System](https://mlsanity.com/paper/2508.15990/) (2025)
- [Self-Supervised Learning for Interactive Perception of Surgical Thread for Autonomous Suture Tail-Shortening](https://mlsanity.com/paper/2307.06845/) (2023)
- [Gentle Object Retraction in Dense Clutter Using Multimodal Force Sensing and Imitation Learning](https://mlsanity.com/paper/2508.19476/) (2025)
- [Towards Robust 3D Object Recognition with Dense-to-Sparse Deep Domain Adaptation](https://mlsanity.com/paper/2205.03654/) (2022)
- [Advancements in Tactile Hand Gesture Recognition for Enhanced Human-Machine Interaction](https://mlsanity.com/paper/2405.17038/) (2024)

## Abstract (arXiv)

This work presents a novel active visuo-tactile based framework for robotic systems to accurately estimate pose of objects in dense cluttered environments. The scene representation is derived using a novel declutter graph (DG) which describes the relationship among objects in the scene for decluttering by leveraging semantic segmentation and grasp affordances networks. The graph formulation allows robots to efficiently declutter the workspace by autonomously selecting the next best object to remove and the optimal action (prehensile or non-prehensile) to perform. Furthermore, we propose a novel translation-invariant Quaternion filter (TIQF) for active vision and active tactile based pose estimation. Both active visual and active tactile points are selected by maximizing the expected information gain. We evaluate our proposed framework on a system with two robots coordinating on randomized scenes of dense cluttered objects and perform ablation studies with static vision and active vision based estimation prior and post decluttering as baselines. Our proposed active visuo-tactile interactive perception framework shows upto 36% improvement in pose accuracy compared to the active vision baseline.