mlsanityData to 7 Oct 2026

Symphonize 3D Semantic Scene Completion with Contextual Instance Queries

arXiv 2306.15670 ↗First posted 27 Jun 2023cs.CV cs.ROcited by 88

Not on the map. Why?

Abstract (arXiv)

`3D Semantic Scene Completion (SSC) has emerged as a nascent and pivotal undertaking in autonomous driving, aiming to predict voxel occupancy within volumetric scenes. However, prevailing methodologies primarily focus on voxel-wise feature aggregation, while neglecting instance semantics and scene context. In this paper, we present a novel paradigm termed Symphonies (Scene-from-Insts), that delves into the integration of instance queries to orchestrate 2D-to-3D reconstruction and 3D scene modeling. Leveraging our proposed Serial Instance-Propagated Attentions, Symphonies dynamically encodes instance-centric semantics, facilitating intricate interactions between image-based and volumetric domains. Simultaneously, Symphonies enables holistic scene comprehension by capturing context through the efficient fusion of instance queries, alleviating geometric ambiguity such as occlusion and perspective errors through contextual scene reasoning. Experimental results demonstrate that Symphonies achieves state-of-the-art performance on challenging benchmarks SemanticKITTI and SSCBench-KITTI-360, yielding remarkable mIoU scores of 15.04 and 18.58, respectively. These results showcase the paradigm's promising advancements. The code is available at https://github.com/hustvl/Symphonies.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Builds on 45 of its 54 references that are in the map. First posted on arXiv 27 Jun 2023.

Builds on

45 of 54 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  10. 2022Planning-oriented Autonomous Drivingcited by 1,068
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
33 more
  1. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  2. 2019Panoptic Feature Pyramid Networkscited by 644
  3. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  4. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  5. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  6. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  7. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  8. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  9. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  10. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  11. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  12. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  13. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  14. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  15. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  16. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  17. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  18. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  19. 2023Scene as Occupancycited by 208
  20. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  21. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  22. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 126
  23. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  24. 2021MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matchingcited by 82
  25. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  26. 20213D Semantic Scene Completion: a Surveycited by 74
  27. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  28. 2019RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completioncited by 54
  29. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  30. 2019Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completioncited by 43
  31. 2019ForkNet: Multi-branch Volumetric Semantic Completion from a Single Depth Imagecited by 31
  32. 2018View-volume Network for Semantic Scene Completion from a Single Depth Imagecited by 26
  33. 2022Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 26; 2025: 42; 2026: 16.

88 papers.

Sort:

By area:

  1. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  2. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  3. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  4. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  5. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  6. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  7. 2023Camera-based 3D Semantic Scene Completion with Sparse Guidance Networkcited by 40
  8. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 39
  9. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  10. 2024Occupancy as Set of Pointscited by 37
  11. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  12. 2024Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehiclescited by 33
38 more
  1. 2024OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Predictioncited by 32
  2. 2023DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptationcited by 30
  3. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  4. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 28
  5. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  6. 2024OccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flowcited by 22
  7. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 20
  8. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 17
  9. 2024LangOcc: Self-Supervised Open Vocabulary Occupancy Estimation via Volume Renderingcited by 16
  10. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  11. 2024Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Drivingcited by 12
  12. 2025VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detectioncited by 12
  13. 2024OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexitycited by 9
  14. 20253D Occupancy Prediction with Low-Resolution Queries via Prototype-aware View Transformationcited by 9
  15. 2024FIction: 4D Future Interaction Prediction from Videocited by 8
  16. 2025L2COcc: Lightweight Camera-Centric Semantic Scene Completion via Distillation of LiDAR Modelcited by 8
  17. 2025VisHall3D: Monocular Semantic Scene Completion from Reconstructing the Visible Regions to Hallucinating the Invisible Regionscited by 7
  18. 2025SplatSSC: Decoupled Depth-Guided Gaussian Splatting for Semantic Scene Completioncited by 7
  19. 2024ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learningcited by 6
  20. 2024LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Predictioncited by 6
  21. 2025VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completioncited by 6
  22. 2025Disentangling Instance and Scene Contexts for 3D Semantic Scene Completioncited by 6
  23. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  24. 2025OccuFly: A 3D Vision Benchmark for Semantic Scene Completion from the Aerial Perspectivecited by 5
  25. 2024Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completioncited by 4
  26. 2024Multi-modal and Multi-scale Spatial Environment Understanding for Immersive Visual Text-to-Speechcited by 4
  27. 2025ST-GS: Vision-Based 3D Semantic Occupancy Prediction with Spatial-Temporal Gaussian Splattingcited by 4
  28. 2025Uni-Inter: Unifying 3D Human Motion Synthesis Across Diverse Interaction Contextscited by 4
  29. 2025VG3T: Visual Geometry Grounded Gaussian Transformercited by 4
  30. 2024ET-Former: Efficient Triplane Deformable Attention for 3D Semantic Scene Completion From Monocular Cameracited by 3
  31. 2025Event-aided Semantic Scene Completioncited by 3
  32. 2025Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidancecited by 3
  33. 2025SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completioncited by 3
  34. 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposalscited by 3
  35. 2025One Step Closer: Creating the Future to Boost Monocular Semantic Scene Completioncited by 3
  36. 2025GaussianFusionOcc: A Seamless Sensor Fusion Approach for 3D Occupancy Prediction Using 3D Gaussianscited by 3
  37. 2025Enhancing 3D Semantic Scene Completion with a Refinement Modulecited by 3
  38. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3

50 most cited of 88 All 88 (JSON)