# Why semantics matters: A deep study on semantic particle-filtering localization in a LiDAR semantic pole-map

arXiv: https://arxiv.org/abs/2305.14038 · HTML: https://mlsanity.com/paper/2305.14038/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 79 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [Mix3D: Out-of-Context Data Augmentation for 3D Scenes](https://mlsanity.com/paper/2110.02210/) (2021)
- [Semantic Visual Localization](https://mlsanity.com/paper/1712.05773/) (2017)
- [Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation](https://mlsanity.com/paper/1812.06707/) (2018)
- [FIDNet: LiDAR Point Cloud Semantic Segmentation with Fully Interpolation Decoding](https://mlsanity.com/paper/2109.03787/) (2021)
- [SegMap: Segment-based mapping and localization using data-driven descriptors](https://mlsanity.com/paper/1909.12837/) (2019)
- [LiDAR Iris for Loop-Closure Detection](https://mlsanity.com/paper/1912.03825/) (2019)
- [Intensity Scan Context: Coding Intensity and Geometry Relations for Loop Closure Detection](https://mlsanity.com/paper/2003.05656/) (2020)
- [RoadMap: A Light-Weight Semantic Map for Visual Localization towards Autonomous Driving](https://mlsanity.com/paper/2106.02527/) (2021)
- [Robust and Precise Vehicle Localization based on Multi-sensor Fusion in Diverse City Scenes](https://mlsanity.com/paper/1711.05805/) (2017)
- [Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization](https://mlsanity.com/paper/1908.03274/) (2019)
- [Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences](https://mlsanity.com/paper/2004.00740/) (2020)
- [Any Way You Look At It: Semantic Crossview Localization and Mapping with LiDAR](https://mlsanity.com/paper/2203.08925/) (2022)
- [Online Pole Segmentation on Range Images for Long-term LiDAR Localization in Urban Environments](https://mlsanity.com/paper/2208.07364/) (2022)
- [Long-Term Localization using Semantic Cues in Floor Plan Maps](https://mlsanity.com/paper/2210.01456/) (2022)
- [Long-Term Urban Vehicle Localization Using Pole Landmarks Extracted from 3-D Lidar Scans](https://mlsanity.com/paper/1910.10550/) (2019)
- [Learning an Overlap-based Observation Model for 3D LiDAR Localization](https://mlsanity.com/paper/2105.11717/) (2021)
- [Online Range Image-based Pole Extractor for Long-term LiDAR Localization in Urban Environments](https://mlsanity.com/paper/2108.08621/) (2021)
- [GMMLoc: Structure Consistent Visual Localization with Gaussian Mixture Models](https://mlsanity.com/paper/2006.13670/) (2020)
- [Seeing the Wood for the Trees: Reliable Localization in Urban and Natural Environments](https://mlsanity.com/paper/1809.02846/) (2018)
- [Pole-like Objects Mapping and Long-Term Robot Localization in Dynamic Urban Scenarios](https://mlsanity.com/paper/2103.13224/) (2021)

## Built on by (most cited first, 2 of 2)

- [Zero-Shot 4D Lidar Panoptic Segmentation](https://mlsanity.com/paper/2504.00848/) (2025)
- [Semantic-Aware Particle Filter for Reliable Vineyard Robot Localisation](https://mlsanity.com/paper/2509.18342/) (2025)

## Abstract (arXiv)

In most urban and suburban areas, pole-like structures such as tree trunks or utility poles are ubiquitous. These structural landmarks are very useful for the localization of autonomous vehicles given their geometrical locations in maps and measurements from sensors. In this work, we aim at creating an accurate map for autonomous vehicles or robots with pole-like structures as the dominant localization landmarks, hence called pole-map. In contrast to the previous pole-based mapping or localization methods, we exploit the semantics of pole-like structures. Specifically, semantic segmentation is achieved by a new mask-range transformer network in a mask-classfication paradigm. With the semantics extracted for the pole-like structures in each frame, a multi-layer semantic pole-map is created by aggregating the detected pole-like structures from all frames. Given the semantic pole-map, we propose a semantic particle-filtering localization scheme for vehicle localization. Theoretically, we have analyzed why the semantic information can benefit the particle-filter localization, and empirically it is validated on the public SemanticKITTI dataset that the particle-filtering localization with semantics achieves much better performance than the counterpart without semantics when each particle's odometry prediction and/or the online observation is subject to uncertainties at significant levels.