mlsanityData to 7 Oct 2026

LidarMultiNet: Unifying LiDAR Semantic Segmentation, 3D Object Detection, and Panoptic Segmentation in a Single Multi-task Network

arXiv 2206.11428 ↗First posted 23 Jun 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

This technical report presents the 1st place winning solution for the Waymo Open Dataset 3D semantic segmentation challenge 2022. Our network, termed LidarMultiNet, unifies the major LiDAR perception tasks such as 3D semantic segmentation, object detection, and panoptic segmentation in a single framework. At the core of LidarMultiNet is a strong 3D voxel-based encoder-decoder network with a novel Global Context Pooling (GCP) module extracting global contextual features from a LiDAR frame to complement its local features. An optional second stage is proposed to refine the first-stage segmentation or generate accurate panoptic segmentation results. Our solution achieves a mIoU of 71.13 and is the best for most of the 22 classes on the Waymo 3D semantic segmentation test set, outperforming all the other 3D semantic segmentation methods on the official leaderboard. We demonstrate for the first time that major LiDAR perception tasks can be unified in a single strong network that can be trained end-to-end.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 23 Jun 2022.

Builds on

30 of 38 references.

References · 30

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  9. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  10. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  11. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  12. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
18 more
  1. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  2. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  3. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  4. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  5. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  6. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  7. 2018A2-Nets: Double Attention Networkscited by 136
  8. 2021LiDAR R-CNN: An Efficient and Universal 3D Object Detectorcited by 129
  9. 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
  10. 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
  11. 2016MultiNet: Real-time Joint Semantic Reasoning for Autonomous Drivingcited by 90
  12. 2020LiDAR-based Panoptic Segmentation via Dynamic Shifting Networkcited by 84
  13. 2019ACFNet: Attentional Class Feature Network for Semantic Segmentationcited by 75
  14. 2021GP-S3Net: Graph-based Panoptic Sparse Semantic Segmentation Networkcited by 37
  15. 2021DRINet: A Dual-Representation Iterative Learning Network for Point Cloud Segmentationcited by 22
  16. 2017A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentationcited by 20
  17. 2015Iterative Instance Segmentationcited by 19
  18. 2021A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understandingcited by 17

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 1; 2025: 1.

8 papers.

  1. 2022LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perceptioncited by 74
  2. 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 72
  3. 2023TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUscited by 36
  4. 2023Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detectioncited by 28
  5. 2023PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformercited by 21
  6. 2024RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentationcited by 15
  7. 2025NUC-Net: Non-uniform Cylindrical Partition Network for Efficient LiDAR Semantic Segmentationcited by 5
  8. 2023RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Modelcited by 3