# Soft Tracking Using Contacts for Cluttered Objects to Perform Blind Object Retrieval

arXiv: https://arxiv.org/abs/2201.10434 · HTML: https://mlsanity.com/paper/2201.10434/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2022.

## Built on by (most cited first, 5 of 5)

- [CHSEL: Producing Diverse Plausible Pose Estimates from Contact and Free Space Data](https://mlsanity.com/paper/2305.08042/) (2023)
- [Gentle Object Retraction in Dense Clutter Using Multimodal Force Sensing and Imitation Learning](https://mlsanity.com/paper/2508.19476/) (2025)
- [Physics-Based Object 6D-Pose Estimation during Non-Prehensile Manipulation](https://mlsanity.com/paper/2211.13572/) (2022)
- [RUMI: Rummaging Using Mutual Information](https://mlsanity.com/paper/2408.10450/) (2024)
- [BEAC: Imitating Complex Exploration and Task-oriented Behaviors for Invisible Object Nonprehensile Manipulation](https://mlsanity.com/paper/2503.16803/) (2025)

## Abstract (arXiv)

Retrieving an object from cluttered spaces suchas cupboards, refrigerators, or bins requires tracking objects with limited or no visual sensing. In these scenarios, contact feedback is necessary to estimate the pose of the objects, yet the objects are movable while their shapes and number may be unknown, making the association of contacts with objects extremely difficult. While previous work has focused on multi-target tracking, the assumptions therein prohibit using prior methods given only the contact-sensing modality. Instead, this paper proposes the method Soft Tracking Using Contacts for Cluttered Objects (STUCCO) that tracks the belief over contact point locations and implicit object associations using a particle filter. This method allows ambiguous object associations of past contacts to be revised as new information becomes available. We apply STUCCO to the Blind Object Retrieval problem, where a target object of known shape but unknown pose must be retrieved from clutter. Our results suggest that our method outperforms baselines in four simulation environments, and on a real robot, where contact sensing is noisy. In simulation, we achieve grasp success of at least 65% on all environments while no baselines achieve over 5%.