# Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center

arXiv: https://arxiv.org/abs/2406.10527 · HTML: https://mlsanity.com/paper/2406.10527/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation](https://mlsanity.com/paper/2307.01492/) (2023)
- [EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI](https://mlsanity.com/paper/2312.16170/) (2023)
- [RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision](https://mlsanity.com/paper/2309.09502/) (2023)
- [PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation](https://mlsanity.com/paper/2306.10013/) (2023)
- [FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin](https://mlsanity.com/paper/2311.12058/) (2023)
- [SSAP: Single-Shot Instance Segmentation With Affinity Pyramid](https://mlsanity.com/paper/1909.01616/) (2019)
- [DeeperLab: Single-Shot Image Parser](https://mlsanity.com/paper/1902.05093/) (2019)
- [Sparse Instance Activation for Real-Time Instance Segmentation](https://mlsanity.com/paper/2203.12827/) (2022)
- [An End-to-End Network for Panoptic Segmentation](https://mlsanity.com/paper/1903.05027/) (2019)
- [MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentation](https://mlsanity.com/paper/2312.06052/) (2023)

## Built on by (most cited first, 14 of 14)

- [ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictions](https://mlsanity.com/paper/2411.07725/) (2024)
- [Deep Height Decoupling for Precise Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2409.07972/) (2024)
- [DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction](https://mlsanity.com/paper/2409.19972/) (2024)
- [Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2504.12959/) (2025)
- [GSRender: Deduplicated Occupancy Prediction via Weakly Supervised 3D Gaussian Splatting](https://mlsanity.com/paper/2412.14579/) (2024)
- [EPRecon: An Efficient Framework for Real-Time Panoptic 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2409.01807/) (2024)
- [Camera-Only 3D Panoptic Scene Completion for Autonomous Driving through Differentiable Object Shapes](https://mlsanity.com/paper/2505.09562/) (2025)
- [MobileOcc: A Human-Aware Semantic Occupancy Dataset for Mobile Robots](https://mlsanity.com/paper/2511.16949/) (2025)
- [SuperOcc: Toward Cohesive Temporal Modeling for Superquadric-based 3D Occupancy Prediction](https://mlsanity.com/paper/2601.15644/) (2026)
- [An Instance-Centric Panoptic Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2603.27238/) (2026)
- [HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Prediction](https://mlsanity.com/paper/2512.19871/) (2025)
- [FreeOcc: Training-free Panoptic Occupancy Prediction via Foundation Models](https://mlsanity.com/paper/2603.06166/) (2026)
- [FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction](https://mlsanity.com/paper/2607.03822/) (2026)
- [OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxels](https://mlsanity.com/paper/2608.29187/) (2026)

## Abstract (arXiv)

Panoptic occupancy poses a novel challenge by aiming to integrate instance occupancy and semantic occupancy within a unified framework. However, there is still a lack of efficient solutions for panoptic occupancy. In this paper, we propose Panoptic-FlashOcc, a straightforward yet robust 2D feature framework that enables realtime panoptic occupancy. Building upon the lightweight design of FlashOcc, our approach simultaneously learns semantic occupancy and class-aware instance clustering in a single network, these outputs are jointly incorporated through panoptic occupancy procession for panoptic occupancy. This approach effectively addresses the drawbacks of high memory and computation requirements associated with three-dimensional voxel-level representations. With its straightforward and efficient design that facilitates easy deployment, Panoptic-FlashOcc demonstrates remarkable achievements in panoptic occupancy prediction. On the Occ3D-nuScenes benchmark, it achieves exceptional performance, with 38.5 RayIoU and 29.1 mIoU for semantic occupancy, operating at a rapid speed of 43.9 FPS. Furthermore, it attains a notable score of 16.0 RayPQ for panoptic occupancy, accompanied by a fast inference speed of 30.2 FPS. These results surpass the performance of existing methodologies in terms of both speed and accuracy. The source code and trained models can be found at the following github repository: https://github.com/Yzichen/FlashOCC.