mlsanityData to 7 Oct 2026

4D Panoptic LiDAR Segmentation

arXiv 2102.12472 ↗First posted 24 Feb 2021cs.CV cs.ROcited by 64

Not on the map. Why?

Abstract (arXiv)

Temporal semantic scene understanding is critical for self-driving cars or robots operating in dynamic environments. In this paper, we propose 4D panoptic LiDAR segmentation to assign a semantic class and a temporally-consistent instance ID to a sequence of 3D points. To this end, we present an approach and a point-centric evaluation metric. Our approach determines a semantic class for every point while modeling object instances as probability distributions in the 4D spatio-temporal domain. We process multiple point clouds in parallel and resolve point-to-instance associations, effectively alleviating the need for explicit temporal data association. Inspired by recent advances in benchmarking of multi-object tracking, we propose to adopt a new evaluation metric that separates the semantic and point-to-instance association aspects of the task. With this work, we aim at paving the road for future developments of temporal LiDAR panoptic perception.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 58 of its 90 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

58 of 90 references.

References · 58

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2017Mask R-CNNcited by 7,259
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  9. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  10. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  11. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  12. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
38 more
  1. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  2. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  3. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  4. 2018Panoptic Segmentationcited by 796
  5. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  6. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 546
  7. 2020Tracking Objects as Pointscited by 483
  8. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  9. 2019Video Instance Segmentationcited by 391
  10. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  11. 2019Tracking without bells and whistlescited by 382
  12. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  13. 2019Panoptic-DeepLabcited by 362
  14. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  15. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  16. 2017SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloudcited by 231
  17. 2018SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloudcited by 228
  18. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  19. 2019Learning a Neural Solver for Multiple Object Trackingcited by 198
  20. 2018Tangent Convolutions for Dense Prediction in 3Dcited by 173
  21. 2017Semantic3D.net: A new Large-scale Point Cloud Classification Benchmarkcited by 158
  22. 2020Video Panoptic Segmentationcited by 141
  23. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  24. 2020MOTChallenge: A Benchmark for Single-Camera Multiple Target Trackingcited by 134
  25. 2017Pointwise Convolutional Neural Networkscited by 126
  26. 2020STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videoscited by 121
  27. 2020GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learningcited by 113
  28. 2019Seamless Scene Segmentationcited by 112
  29. 2019Just Go with the Flow: Self-Supervised Scene Flow Estimationcited by 98
  30. 2019Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidthcited by 88
  31. 2020P2B: Point-to-Box Network for 3D Object Tracking in Point Cloudscited by 84
  32. 2018PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentationcited by 81
  33. 2016Learning by tracking: Siamese CNN for robust target associationcited by 79
  34. 2019Identifying Unknown Instances for Autonomous Drivingcited by 73
  35. 2021STEP: Segmenting and Tracking Every Pixelcited by 72
  36. 2019How To Train Your Deep Multi-Object Trackercited by 71
  37. 2019A-CNN: Annularly Convolutional Neural Networks on Point Cloudscited by 66
  38. 2019FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Trackingcited by 65

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 16; 2023: 12; 2024: 15; 2025: 10; 2026: 8.

64 papers.

Sort:

By area:

  1. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  2. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  3. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  4. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  5. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  6. 2021Opening up Open-World Trackingcited by 50
  7. 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
  8. 2024Better Call SAL: Towards Learning to Segment Anything in Lidarcited by 47
  9. 20224D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregationcited by 35
  10. 2022LidarCLIP or: How I Learned to Talk to Point Cloudscited by 33
  11. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 32
  12. 2023DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Cloudscited by 31
38 more
  1. 2022PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking?cited by 29
  2. 2022Forecasting from LiDAR via Future Object Detectioncited by 28
  3. 2022Domain generalization of 3D semantic segmentation in autonomous drivingcited by 25
  4. 2023UrbanBIS: a Large-scale Benchmark for Fine-grained Urban Building Instance Segmentationcited by 24
  5. 2023Mask4Former: Mask Transformer for 4D Panoptic Segmentationcited by 24
  6. 2022Sparse Cross-scale Attention Network for Efficient LiDAR Panoptic Segmentationcited by 22
  7. 20234D Panoptic Segmentation as Invariant and Equivariant Field Predictioncited by 19
  8. 2022Revealing Occlusions with 4D Neural Fieldscited by 17
  9. 2022Online Segmentation of LiDAR Sequences: Dataset and Algorithmcited by 14
  10. 2022Scalable SoftGroup for 3D Instance Segmentation on Point Cloudscited by 14
  11. 2024Scalable 3D Panoptic Segmentation As Superpoint Graph Clusteringcited by 14
  12. 2023SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentationcited by 13
  13. 2024TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentationcited by 13
  14. 20234D-Former: Multimodal 4D Panoptic Segmentationcited by 12
  15. 2024Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Drivingcited by 12
  16. 2024An Empirical Study of Training State-of-the-Art LiDAR Segmentation Modelscited by 12
  17. 2022LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Networkcited by 11
  18. 2024JRDB-PanoTrack: An Open-world Panoptic Segmentation and Tracking Robotic Dataset in Crowded Human Environmentscited by 10
  19. 2022Unsupervised 4D LiDAR Moving Object Segmentation in Stationary Settings with Multivariate Occupancy Time Seriescited by 8
  20. 2024Multimodal 3D Object Detection on Unseen Domainscited by 8
  21. 2023Lidar Panoptic Segmentation and Tracking without Bells and Whistlescited by 7
  22. 2024Lidar Panoptic Segmentation in an Open Worldcited by 7
  23. 2024ZOPP: A Framework of Zero-shot Offboard Panoptic Perception for Autonomous Drivingcited by 6
  24. 2025Zero-Shot 4D Lidar Panoptic Segmentationcited by 6
  25. 2022Learning Moving-Object Tracking with FMCW LiDARcited by 5
  26. 2022Sequential Point Clouds: A Surveycited by 5
  27. 2024SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloudcited by 5
  28. 2024UNIT: Unsupervised Online Instance Segmentation through Timecited by 5
  29. 2025Open-Set LiDAR Panoptic Segmentation Guided by Uncertainty-Aware Learningcited by 5
  30. 20264DPC2hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrappingcited by 5
  31. 2024Interactive4D: Interactive 4D LiDAR Segmentationcited by 4
  32. 2025SALT: A Flexible Semi-Automatic Labeling Tool for General LiDAR Point Clouds with Cross-Scene Adaptability and 4D Consistencycited by 4
  33. 2023ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Datacited by 3
  34. 2025TrackOcc: Camera-based 4D Panoptic Occupancy Trackingcited by 3
  35. 2025HiLoTs: High-Low Temporal Sensitive Representation Learning for Semi-Supervised LiDAR Segmentation in Autonomous Drivingcited by 3
  36. 2026Latent Gaussian Splatting for 4D Panoptic Occupancy Trackingcited by 3
  37. 2023Multi-to-Single Knowledge Distillation for Point Cloud Semantic Segmentationcited by 2
  38. 2024PCLD: Point Cloud Layerwise Diffusion for Adversarial Purificationcited by 2

50 most cited of 64 All 64 (JSON)