# Volumetric Mapping with Panoptic Refinement via Kernel Density Estimation for Mobile Robots

arXiv: https://arxiv.org/abs/2412.11241 · HTML: https://mlsanity.com/paper/2412.11241/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning](https://mlsanity.com/paper/1611.03631/) (2016)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [Depth-aware CNN for RGB-D Segmentation](https://mlsanity.com/paper/1803.06791/) (2018)
- [Indoor Semantic Segmentation using depth information](https://mlsanity.com/paper/1301.3572/) (2013)
- [Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects](https://mlsanity.com/paper/1706.06629/) (2017)
- [On Advantages of Mask-level Recognition for Outlier-aware Segmentation](https://mlsanity.com/paper/2301.03407/) (2023)
- [Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency](https://mlsanity.com/paper/2109.10165/) (2021)
- [Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks](https://mlsanity.com/paper/1608.01082/) (2016)
- [Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation](https://mlsanity.com/paper/1803.02784/) (2018)
- [Real-time 3D Semantic Scene Perception for Egocentric Robots with Binocular Vision](https://mlsanity.com/paper/2402.11872/) (2024)
- [Dynamic High Resolution Deformable Articulated Tracking](https://mlsanity.com/paper/1711.07999/) (2017)

## Built on by (most cited first, 2 of 2)

- [Distortion-Aware Adversarial Attacks on Bounding Boxes of Object Detectors](https://mlsanity.com/paper/2412.18815/) (2024)
- [Improved 3D Point-Line Mapping Regression for Camera Relocalization](https://mlsanity.com/paper/2502.20814/) (2025)

## Abstract (arXiv)

Reconstructing three-dimensional (3D) scenes with semantic understanding is vital in many robotic applications. Robots need to identify which objects, along with their positions and shapes, to manipulate them precisely with given tasks. Mobile robots, especially, usually use lightweight networks to segment objects on RGB images and then localize them via depth maps; however, they often encounter out-of-distribution scenarios where masks over-cover the objects. In this paper, we address the problem of panoptic segmentation quality in 3D scene reconstruction by refining segmentation errors using non-parametric statistical methods. To enhance mask precision, we map the predicted masks into a depth frame to estimate their distribution via kernel densities. The outliers in depth perception are then rejected without the need for additional parameters in an adaptive manner to out-of-distribution scenarios, followed by 3D reconstruction using projective signed distance functions (SDFs). We validate our method on a synthetic dataset, which shows improvements in both quantitative and qualitative results for panoptic mapping. Through real-world testing, the results furthermore show our method's capability to be deployed on a real-robot system. Our source code is available at: https://github.com/mkhangg/refined panoptic mapping.