# Deep Height Decoupling for Precise Vision-based 3D Occupancy Prediction

arXiv: https://arxiv.org/abs/2409.07972 · HTML: https://mlsanity.com/paper/2409.07972/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 53 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2304.05316/) (2023)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)
- [FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation](https://mlsanity.com/paper/2307.01492/) (2023)
- [GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2405.17429/) (2024)
- [SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2311.12754/) (2023)
- [RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision](https://mlsanity.com/paper/2309.09502/) (2023)
- [PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation](https://mlsanity.com/paper/2306.10013/) (2023)
- [FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin](https://mlsanity.com/paper/2311.12058/) (2023)
- [COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2312.01919/) (2023)
- [OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.13540/) (2023)
- [SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2304.14340/) (2023)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deployment](https://mlsanity.com/paper/2211.17111/) (2022)
- [Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark](https://mlsanity.com/paper/2108.03830/) (2021)
- [FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective View](https://mlsanity.com/paper/2403.02710/) (2024)
- [Context and Geometry Aware Voxel Transformer for Semantic Scene Completion](https://mlsanity.com/paper/2405.13675/) (2024)

## Built on by (most cited first, 16 of 16)

- [Language Driven Occupancy Prediction](https://mlsanity.com/paper/2411.16072/) (2024)
- [DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction](https://mlsanity.com/paper/2409.19972/) (2024)
- [SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queries](https://mlsanity.com/paper/2510.17482/) (2025)
- [VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction](https://mlsanity.com/paper/2506.05563/) (2025)
- [Semantic Causality-Aware Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2509.08388/) (2025)
- [Learning Multi-scale Spatial-frequency Features for Image Denoising](https://mlsanity.com/paper/2506.16307/) (2025)
- [LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2412.05976/) (2024)
- [Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection](https://mlsanity.com/paper/2504.04732/) (2025)
- [See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Prediction](https://mlsanity.com/paper/2505.20641/) (2025)
- [Out-of-Distribution Semantic Occupancy Prediction](https://mlsanity.com/paper/2506.21185/) (2025)
- [Collaborative Learning of Local 3D Occupancy Prediction and Versatile Global Occupancy Mapping](https://mlsanity.com/paper/2504.13596/) (2025)
- [DA-Occ: Direction-Aware 2D Convolution for Efficient and Geometry-Preserving 3D Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2507.23599/) (2025)
- [Height-Guided Projection Reparameterization for Camera-LiDAR Occupancy](https://mlsanity.com/paper/2605.05072/) (2026)
- [FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction](https://mlsanity.com/paper/2607.03822/) (2026)
- [DualPathOcc: Dual-Resolution BEV Encoder for 3D Occupancy Prediction](https://mlsanity.com/paper/2609.06370/) (2026)
- [Risk-Aware Occupancy for Safety-Oriented End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.21470/) (2026)

## Abstract (arXiv)

The task of vision-based 3D occupancy prediction aims to reconstruct 3D geometry and estimate its semantic classes from 2D color images, where the 2D-to-3D view transformation is an indispensable step. Most previous methods conduct forward projection, such as BEVPooling and VoxelPooling, both of which map the 2D image features into 3D grids. However, the current grid representing features within a certain height range usually introduces many confusing features that belong to other height ranges. To address this challenge, we present Deep Height Decoupling (DHD), a novel framework that incorporates explicit height prior to filter out the confusing features. Specifically, DHD first predicts height maps via explicit supervision. Based on the height distribution statistics, DHD designs Mask Guided Height Sampling (MGHS) to adaptively decouple the height map into multiple binary masks. MGHS projects the 2D image features into multiple subspaces, where each grid contains features within reasonable height ranges. Finally, a Synergistic Feature Aggregation (SFA) module is deployed to enhance the feature representation through channel and spatial affinities, enabling further occupancy refinement. On the popular Occ3D-nuScenes benchmark, our method achieves state-of-the-art performance even with minimal input frames. Source code is released at https://github.com/yanzq95/DHD.