mlsanityData to 7 Oct 2026

PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness

arXiv 2312.02158 ↗First posted 4 Dec 2023cs.CV cs.AIcited by 42

Not on the map. Why?

Abstract (arXiv)

We propose the task of Panoptic Scene Completion (PSC) which extends the recently popular Semantic Scene Completion (SSC) task with instance-level information to produce a richer understanding of the 3D scene. Our PSC proposal utilizes a hybrid mask-based technique on the non-empty voxels from sparse multi-scale completions. Whereas the SSC literature overlooks uncertainty which is critical for robotics applications, we instead propose an efficient ensembling to estimate both voxel-wise and instance-wise uncertainties along PSC. This is achieved by building on a multi-input multi-output (MIMO) strategy, while improving performance and yielding better uncertainty for little additional compute. Additionally, we introduce a technique to aggregate permutation-invariant mask predictions. Our experiments demonstrate that our method surpasses all baselines in both Panoptic Scene Completion and uncertainty estimation on three large-scale autonomous driving datasets. Our code and data are available at https://astra-vision.github.io/PaSCo .

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 63 of its 79 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

63 of 79 references.

References · 63

  1. 2017Mask R-CNNcited by 7,259
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  10. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  11. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  12. 2018Panoptic Segmentationcited by 796
38 more
  1. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  2. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  3. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  4. 2015Weight Uncertainty in Neural Networkscited by 554
  5. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  6. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  7. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  8. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  9. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  10. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  11. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  12. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  13. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  14. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  15. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  16. 2020Training independent subnetworks for robust predictioncited by 177
  17. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  18. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  19. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 126
  20. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  21. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  22. 2019Seamless Scene Segmentationcited by 112
  23. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  24. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  25. 2020Masksembles for Uncertainty Estimationcited by 88
  26. 2020LiDAR-based Panoptic Segmentation via Dynamic Shifting Networkcited by 84
  27. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  28. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  29. 2021EfficientLPS: Efficient LiDAR Panoptic Segmentationcited by 76
  30. 20213D Semantic Scene Completion: a Surveycited by 74
  31. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
  32. 2019Efficient Semantic Scene Completion Network with Spatial Group Convolutioncited by 66
  33. 2022Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic Segmentation via Clustering Pseudo Heatmapcited by 59
  34. 2021Semantic Scene Completion via Integrating Instances and Scene in-the-Loopcited by 54
  35. 2020MOPT: Multi-Object Panoptic Trackingcited by 53
  36. 2023Using a Waffle Iron for Automotive Point Cloud Semantic Segmentationcited by 53
  37. 2022Packed-Ensembles for Efficient Uncertainty Estimationcited by 49
  38. 2018Two Stream 3D Semantic Scene Completioncited by 48

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 18; 2026: 13.

42 papers.

Sort:

By area:

  1. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  2. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 32
  3. 2024Is Your LiDAR Placement Optimized for 3D Scene Understanding?cited by 25
  4. 2024LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Cameracited by 18
  5. 2024ProtoOcc: Accurate, Efficient 3D Occupancy Prediction Using Dual Branch Encoder-Prototype Query Decodercited by 17
  6. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  7. 2024An Empirical Study of Training State-of-the-Art LiDAR Segmentation Modelscited by 12
  8. 2024HybridOcc: NeRF Enhanced Transformer-based Multi-Camera 3D Occupancy Predictioncited by 12
  9. 2025VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detectioncited by 12
  10. 2024Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understandingcited by 11
  11. 2024OccScene: Semantic Occupancy-based Cross-task Mutual Learning for 3D Scene Generationcited by 10
  12. 2025L2COcc: Lightweight Camera-Centric Semantic Scene Completion via Distillation of LiDAR Modelcited by 8
30 more
  1. 2025Occupancy Learning with Spatiotemporal Memorycited by 8
  2. 2025Towards Learning to Complete Anything in Lidarcited by 7
  3. 2025How Do Images Align and Complement LiDAR? Towards a Harmonized Multi-modal 3D Panoptic Segmentationcited by 7
  4. 2025OCCUQ: Exploring Efficient Uncertainty Quantification for 3D Occupancy Predictioncited by 6
  5. 2026OccAny: Generalized Unconstrained Urban 3D Occupancycited by 6
  6. 2025Delta-Triplane Transformers as Occupancy World Modelscited by 5
  7. 2025DOC-Depth: A novel approach for dense depth ground truth generationcited by 3
  8. 2025TrackOcc: Camera-based 4D Panoptic Occupancy Trackingcited by 3
  9. 2025Uncertainty-Instructed Structure Injection for Generalizable HD Map Constructioncited by 3
  10. 2025Audio-Driven Talking Face Video Generation with Joint Uncertainty Learningcited by 3
  11. 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposalscited by 3
  12. 2025Enhancing 3D Semantic Scene Completion with a Refinement Modulecited by 3
  13. 2026Latent Gaussian Splatting for 4D Panoptic Occupancy Trackingcited by 3
  14. 2025Out-of-Distribution Semantic Occupancy Predictioncited by 2
  15. 2025TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completioncited by 2
  16. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  17. 2025Point-based Instance Completion with Scene Constraintscited by 1
  18. 2026An Instance-Centric Panoptic Occupancy Prediction Benchmark for Autonomous Drivingcited by 1
  19. 2024α-OCC: Uncertainty-Aware Camera-based 3D Semantic Occupancy Predictioncited by 0
  20. 2024LASER: Script Execution by Autonomous Agents for On-demand Traffic Simulationcited by 0
  21. 2025ELMAR: Enhancing LiDAR Detection with 4D Radar Motion Awareness and Cross-modal Uncertaintycited by 0
  22. 2026SUG-Occ: Explicit Semantics and Uncertainty Guided Sparse Learning for Efficient 3D Occupancy Predictioncited by 0
  23. 2026Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flowcited by 0
  24. 2026PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Drivingcited by 0
  25. 2026Deformable Gaussian Occupancy: Decoupling Rigid and Nonrigid Motion with Factorized Distillationcited by 0
  26. 2026Streaming Gaussian Encoding for 4D Panoptic Occupancy Trackingcited by 0
  27. 2026GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perceptioncited by 0
  28. 2026Generative Semantic Scene Completioncited by 0
  29. 2026SSC-Priors: Exploring Semantic and Visibility Priors to Boost Lidar Semantic Scene Completioncited by 0
  30. 2026CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Imagecited by 0