mlsanityData to 14 Oct 2026

ConvBKI: Real-Time Probabilistic Semantic Mapping Network with Quantifiable Uncertainty

arXiv 2310.16020 ↗First posted 24 Oct 2023cs.RO cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we develop a modular neural network for real-time colorblack(> 10 Hz) semantic mapping in uncertain environments, which explicitly updates per-voxel probabilistic distributions within a neural network layer. Our approach combines the reliability of classical probabilistic algorithms with the performance and efficiency of modern neural networks. Although robotic perception is often divided between modern differentiable methods and classical explicit methods, a union of both is necessary for real-time and trustworthy performance. We introduce a novel Convolutional Bayesian Kernel Inference (ConvBKI) layer which incorporates semantic segmentation predictions online into a 3D map through a depthwise convolution layer by leveraging conjugate priors. We compare ConvBKI against state-of-the-art deep learning approaches and probabilistic algorithms for mapping to evaluate reliability and performance. We also create a Robot Operating System (ROS) package of ConvBKI and test it on real-world perceptually challenging off-road driving data.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 38 of its 73 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

38 of 73 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  12. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
26 more
  1. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  2. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  3. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  4. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  5. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  6. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  7. 2020LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mappingcited by 188
  8. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 184
  9. 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
  10. 2016Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planningcited by 150
  11. 2020RELLIS-3D Dataset: Data, Benchmarks and Analysiscited by 119
  12. 2021Translating Images into Mapscited by 115
  13. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  14. 2021SuMa++: Efficient LiDAR-based Semantic SLAMcited by 100
  15. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 58
  16. 2022MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environmentscited by 40
  17. 2021Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistencycited by 39
  18. 2019Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mappingcited by 35
  19. 2017DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networkscited by 18
  20. 2018Recurrent-OctoMap: Learning State-based Map Refinement for Long-Term Semantic Mapping with 3D-Lidar Datacited by 16
  21. 2017Semantic 3D Occupancy Mapping through Efficient High Order CRFscited by 15
  22. 2020The continuous categorical: a novel simplex-valued exponential familycited by 12
  23. 2022These Maps Are Made For Walking: Real-Time Terrain Property Estimation for Mobile Robotscited by 12
  24. 2022Convolutional Bayesian Kernel Inference for 3D Semantic Mappingcited by 8
  25. 2017Gaussian Processes Semantic Map Representationcited by 6
  26. 2021Dynamic Semantic Occupancy Mapping using 3D Scene Flow and Closed-Form Bayesian Inferencecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 3.

11 papers.

  1. 2025POp-GS: Next Best View in 3D-Gaussian Splatting with P-Optimalitycited by 18
  2. 2024Modeling Uncertainty in 3D Gaussian Splatting through Continuous Semantic Splattingcited by 12
  3. 2024Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inferencecited by 6
  4. 2025These Magic Moments: Differentiable Uncertainty Quantification of Radiance Field Modelscited by 6
  5. 2025Understanding while Exploring: Semantics-driven Active Mappingcited by 5
  6. 2024Uncertainty-aware Semantic Mapping in Off-road Environments with Dempster-Shafer Theory of Evidencecited by 2
  7. 2024LatentBKI: Open-Dictionary Continuous Mapping in Visual-Language Latent Spaces with Quantifiable Uncertaintycited by 2
  8. 2025SLIM-VDB: A Real-Time 3D Probabilistic Semantic Mapping Frameworkcited by 0
  9. 2026Belief Consistency Between Foundation-Model Evidence and Geometric Perception in Persistent Robotic Mapscited by 0
  10. 2026VoxelFix: Post-Hoc Semantic Correction of Completed 3D Voxel Mapscited by 0
  11. 2026DIGS: Distribution-Informed Gaussian Splatting for Training-Free Open-Vocabulary 3D Segmentationcited by 0