# Map Space Belief Prediction for Manipulation-Enhanced Mapping

arXiv: https://arxiv.org/abs/2502.20606 · HTML: https://mlsanity.com/paper/2502.20606/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework](https://mlsanity.com/paper/2205.13790/) (2022)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation](https://mlsanity.com/paper/2110.03051/) (2021)
- [Learning to Map for Active Semantic Goal Navigation](https://mlsanity.com/paper/2106.15648/) (2021)
- [Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter](https://mlsanity.com/paper/1903.01588/) (2019)
- [PEANUT: Predicting and Navigating to Unseen Targets](https://mlsanity.com/paper/2212.02497/) (2022)
- [Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility](https://mlsanity.com/paper/1806.10354/) (2018)
- [On the Overconfidence Problem in Semantic 3D Mapping](https://mlsanity.com/paper/2311.10018/) (2023)
- [Mechanical Search on Shelves using Lateral Access X-RAY](https://mlsanity.com/paper/2011.11696/) (2020)
- [Viewpoint Push Planning for Mapping of Unknown Confined Spaces](https://mlsanity.com/paper/2303.03126/) (2023)
- [Learning to Gather Information via Imitation](https://mlsanity.com/paper/1611.04180/) (2016)
- [Active Implicit Reconstruction Using One-Shot View Planning](https://mlsanity.com/paper/2310.00685/) (2023)
- [Visibility-Aware Navigation Among Movable Obstacles](https://mlsanity.com/paper/2212.02671/) (2022)
- [Planning for Manipulation among Movable Objects: Deciding Which Objects Go Where, in What Order, and How](https://mlsanity.com/paper/2303.13385/) (2023)

## Built on by (most cited first, 1 of 1)

- [EvidMTL: Evidential Multi-Task Learning for Uncertainty-Aware Semantic Surface Mapping from Monocular RGB Images](https://mlsanity.com/paper/2503.04441/) (2025)

## Abstract (arXiv)

Searching for objects in cluttered environments requires selecting efficient viewpoints and manipulation actions to remove occlusions and reduce uncertainty in object locations, shapes, and categories. In this work, we address the problem of manipulation-enhanced semantic mapping, where a robot has to efficiently identify all objects in a cluttered shelf. Although Partially Observable Markov Decision Processes~(POMDPs) are standard for decision-making under uncertainty, representing unstructured interactive worlds remains challenging in this formalism. To tackle this, we define a POMDP whose belief is summarized by a metric-semantic grid map and propose a novel framework that uses neural networks to perform map-space belief updates to reason efficiently and simultaneously about object geometries, locations, categories, occlusions, and manipulation physics. Further, to enable accurate information gain analysis, the learned belief updates should maintain calibrated estimates of uncertainty. Therefore, we propose Calibrated Neural-Accelerated Belief Updates (CNABUs) to learn a belief propagation model that generalizes to novel scenarios and provides confidence-calibrated predictions for unknown areas. Our experiments show that our novel POMDP planner improves map completeness and accuracy over existing methods in challenging simulations and successfully transfers to real-world cluttered shelves in zero-shot fashion.