# RUMI: Rummaging Using Mutual Information

arXiv: https://arxiv.org/abs/2408.10450 · HTML: https://mlsanity.com/paper/2408.10450/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 19 of 19)

- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking](https://mlsanity.com/paper/1905.09304/) (2019)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2209.08409/) (2022)
- [Active 3D Shape Reconstruction from Vision and Touch](https://mlsanity.com/paper/2107.09584/) (2021)
- [MidasTouch: Monte-Carlo inference over distributions across sliding touch](https://mlsanity.com/paper/2210.14210/) (2022)
- [Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter](https://mlsanity.com/paper/1903.01588/) (2019)
- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [Soft-Bubble grippers for robust and perceptive manipulation](https://mlsanity.com/paper/2004.03691/) (2020)
- [Active Visuo-Haptic Object Shape Completion](https://mlsanity.com/paper/2203.09149/) (2022)
- [A pressure field model for fast, robust approximation of net contact force and moment between nominally rigid objects](https://mlsanity.com/paper/1904.11433/) (2019)
- [Informative Path Planning for Active Field Mapping under Localization Uncertainty](https://mlsanity.com/paper/1902.09660/) (2019)
- [Multi-Robot Informative Path Planning for Active Sensing of Environmental Phenomena: A Tale of Two Algorithms](https://mlsanity.com/paper/1302.0723/) (2013)
- [Velocity Level Approximation of Pressure Field Contact Patches](https://mlsanity.com/paper/2110.04157/) (2021)
- [Soft Tracking Using Contacts for Cluttered Objects to Perform Blind Object Retrieval](https://mlsanity.com/paper/2201.10434/) (2022)
- [CHSEL: Producing Diverse Plausible Pose Estimates from Contact and Free Space Data](https://mlsanity.com/paper/2305.08042/) (2023)
- [Active Exploration Using Gaussian Random Fields and Gaussian Process Implicit Surfaces](https://mlsanity.com/paper/1802.04642/) (2018)
- [Spline-Interpolated Model Predictive Path Integral Control with Stein Variational Inference for Reactive Navigation](https://mlsanity.com/paper/2404.10395/) (2024)

## Abstract (arXiv)

This paper presents Rummaging Using Mutual Information (RUMI), a method for online generation of robot action sequences to gather information about the pose of a known movable object in visually-occluded environments. Focusing on contact-rich rummaging, our approach leverages mutual information between the object pose distribution and robot trajectory for action planning. From an observed partial point cloud, RUMI deduces the compatible object pose distribution and approximates the mutual information of it with workspace occupancy in real time. Based on this, we develop an information gain cost function and a reachability cost function to keep the object within the robot's reach. These are integrated into a model predictive control (MPC) framework with a stochastic dynamics model, updating the pose distribution in a closed loop. Key contributions include a new belief framework for object pose estimation, an efficient information gain computation strategy, and a robust MPC-based control scheme. RUMI demonstrates superior performance in both simulated and real tasks compared to baseline methods.