# Neural Semantic Map-Learning for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2410.07780 · HTML: https://mlsanity.com/paper/2410.07780/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 26 of 26)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [Urban Radiance Fields](https://mlsanity.com/paper/2111.14643/) (2021)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning](https://mlsanity.com/paper/1611.03631/) (2016)
- [SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations](https://mlsanity.com/paper/2210.02299/) (2022)
- [NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping](https://mlsanity.com/paper/2303.10709/) (2023)
- [RoadMap: A Light-Weight Semantic Map for Visual Localization towards Autonomous Driving](https://mlsanity.com/paper/2106.02527/) (2021)
- [Voxgraph: Globally Consistent, Volumetric Mapping using Signed Distance Function Submaps](https://mlsanity.com/paper/2004.13154/) (2020)
- [Real-time Scalable Dense Surfel Mapping](https://mlsanity.com/paper/1909.04250/) (2019)
- [Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping](https://mlsanity.com/paper/2011.04087/) (2020)
- [maplab 2.0 -- A Modular and Multi-Modal Mapping Framework](https://mlsanity.com/paper/2212.00654/) (2022)
- [ORBSLAM-Atlas: a robust and accurate multi-map system](https://mlsanity.com/paper/1908.11585/) (2019)
- [Collaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation](https://mlsanity.com/paper/1801.08361/) (2018)
- [Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications](https://mlsanity.com/paper/2203.01087/) (2022)
- [An End-to-End System for Crowdsourced 3d Maps for Autonomous Vehicles: The Mapping Component](https://mlsanity.com/paper/1703.10193/) (2017)
- [Large-Scale 3D Semantic Reconstruction for Automated Driving Vehicles with Adaptive Truncated Signed Distance Function](https://mlsanity.com/paper/2202.13855/) (2022)

## Built on by (most cited first, 1 of 1)

- [FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving](https://mlsanity.com/paper/2505.00318/) (2025)

## Abstract (arXiv)

Autonomous vehicles demand detailed maps to maneuver reliably through traffic, which need to be kept up-to-date to ensure a safe operation. A promising way to adapt the maps to the ever-changing road-network is to use crowd-sourced data from a fleet of vehicles. In this work, we present a mapping system that fuses local submaps gathered from a fleet of vehicles at a central instance to produce a coherent map of the road environment including drivable area, lane markings, poles, obstacles and more as a 3D mesh. Each vehicle contributes locally reconstructed submaps as lightweight meshes, making our method applicable to a wide range of reconstruction methods and sensor modalities. Our method jointly aligns and merges the noisy and incomplete local submaps using a scene-specific Neural Signed Distance Field, which is supervised using the submap meshes to predict a fused environment representation. We leverage memory-efficient sparse feature-grids to scale to large areas and introduce a confidence score to model uncertainty in scene reconstruction. Our approach is evaluated on two datasets with different local mapping methods, showing improved pose alignment and reconstruction over existing methods. Additionally, we demonstrate the benefit of multi-session mapping and examine the required amount of data to enable high-fidelity map learning for autonomous vehicles.