# Learning Occupancy for Monocular 3D Object Detection

arXiv: https://arxiv.org/abs/2305.15694 · HTML: https://mlsanity.com/paper/2305.15694/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 72 of its 83 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 30 of 72)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)

## Built on by (most cited first, 30 of 36)

- [FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin](https://mlsanity.com/paper/2311.12058/) (2023)
- [A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspective](https://mlsanity.com/paper/2405.05173/) (2024)
- [Vision-based 3D occupancy prediction in autonomous driving: a review and outlook](https://mlsanity.com/paper/2405.02595/) (2024)
- [STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Prediction](https://mlsanity.com/paper/2504.19749/) (2025)
- [OccFlowNet: Towards Self-supervised Occupancy Estimation via Differentiable Rendering and Occupancy Flow](https://mlsanity.com/paper/2402.12792/) (2024)
- [MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors](https://mlsanity.com/paper/2410.19590/) (2024)
- [MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders](https://mlsanity.com/paper/2405.07696/) (2024)
- [Deep Height Decoupling for Precise Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2409.07972/) (2024)
- [DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction](https://mlsanity.com/paper/2409.19972/) (2024)
- [Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving](https://mlsanity.com/paper/2403.08504/) (2024)
- [CARLA Drone: Monocular 3D Object Detection from a Different Perspective](https://mlsanity.com/paper/2408.11958/) (2024)
- [Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction](https://mlsanity.com/paper/2602.21552/) (2026)
- [MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection](https://mlsanity.com/paper/2404.04910/) (2024)
- [Unleashing the Power of Chain-of-Prediction for Monocular 3D Object Detection](https://mlsanity.com/paper/2505.04594/) (2025)
- [VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction](https://mlsanity.com/paper/2506.05563/) (2025)
- [TEOcc: Radar-camera Multi-modal Occupancy Prediction via Temporal Enhancement](https://mlsanity.com/paper/2410.11228/) (2024)
- [Spatial Retrieval Augmented Autonomous Driving](https://mlsanity.com/paper/2512.06865/) (2025)
- [Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection](https://mlsanity.com/paper/2504.04732/) (2025)
- [Towards Intrinsic-Aware Monocular 3D Object Detection](https://mlsanity.com/paper/2603.27059/) (2026)
- [BEV-IO: Enhancing Bird's-Eye-View 3D Detection with Instance Occupancy](https://mlsanity.com/paper/2305.16829/) (2023)
- [DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR](https://mlsanity.com/paper/2604.03685/) (2026)
- [Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Data](https://mlsanity.com/paper/2411.15657/) (2024)
- [IDEAL-M3D: Instance Diversity-Enriched Active Learning for Monocular 3D Detection](https://mlsanity.com/paper/2511.19301/) (2025)
- [UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height](https://mlsanity.com/paper/2409.11160/) (2024)
- [LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection](https://mlsanity.com/paper/2512.05663/) (2025)
- [S3-MonoDETR: Supervised Shape&Scale-perceptive Deformable Transformer for Monocular 3D Object Detection](https://mlsanity.com/paper/2309.00928/) (2023)
- [Revisiting Monocular 3D Object Detection with Depth Thickness Field](https://mlsanity.com/paper/2412.19165/) (2024)
- [MonoVQD: Monocular 3D Object Detection with Variational Query Denoising and Self-Distillation](https://mlsanity.com/paper/2506.14835/) (2025)
- [StixelNExT++: Lightweight Monocular Scene Segmentation and Representation for Collective Perception](https://mlsanity.com/paper/2507.06687/) (2025)
- [Mono3DV: Monocular 3D Object Detection with 3D-Aware Bipartite Matching and Variational Query DeNoising](https://mlsanity.com/paper/2601.01036/) (2026)

## Abstract (arXiv)

Monocular 3D detection is a challenging task due to the lack of accurate 3D information. Existing approaches typically rely on geometry constraints and dense depth estimates to facilitate the learning, but often fail to fully exploit the benefits of three-dimensional feature extraction in frustum and 3D space. In this paper, we propose OccupancyM3D, a method of learning occupancy for monocular 3D detection. It directly learns occupancy in frustum and 3D space, leading to more discriminative and informative 3D features and representations. Specifically, by using synchronized raw sparse LiDAR point clouds, we define the space status and generate voxel-based occupancy labels. We formulate occupancy prediction as a simple classification problem and design associated occupancy losses. Resulting occupancy estimates are employed to enhance original frustum/3D features. As a result, experiments on KITTI and Waymo open datasets demonstrate that the proposed method achieves a new state of the art and surpasses other methods by a significant margin. Codes and pre-trained models will be available at urlhttps://github.com/SPengLiang/OccupancyM3D.