# BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

arXiv: https://arxiv.org/abs/2303.08498 · HTML: https://mlsanity.com/paper/2303.08498/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [Disentangling Monocular 3D Object Detection](https://mlsanity.com/paper/1905.12365/) (2019)
- [Objects are Different: Flexible Monocular 3D Object Detection](https://mlsanity.com/paper/2104.02323/) (2021)
- [Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection](https://mlsanity.com/paper/2210.02443/) (2022)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [PolarFormer: Multi-camera 3D Object Detection with Polar Transformer](https://mlsanity.com/paper/2206.15398/) (2022)
- [Delving into Localization Errors for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.16237/) (2021)
- [Kinematic 3D Object Detection in Monocular Video](https://mlsanity.com/paper/2007.09548/) (2020)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformer](https://mlsanity.com/paper/2203.10981/) (2022)

## Built on by (most cited first, 30 of 75)

- [OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving](https://mlsanity.com/paper/2405.20337/) (2024)
- [TUMTraf V2X Cooperative Perception Dataset](https://mlsanity.com/paper/2403.01316/) (2024)
- [End-to-End Autonomous Driving through V2X Cooperation](https://mlsanity.com/paper/2404.00717/) (2024)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)
- [RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception](https://mlsanity.com/paper/2403.10145/) (2024)
- [GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2310.08261/) (2023)
- [GraphBEV: Towards Robust BEV Feature Alignment for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2403.11848/) (2024)
- [Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autonomous Driving](https://mlsanity.com/paper/2310.07602/) (2023)
- [Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2311.01682/) (2023)
- [BEVHeight++: Toward Robust Visual Centric 3D Object Detection](https://mlsanity.com/paper/2309.16179/) (2023)
- [MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings](https://mlsanity.com/paper/2310.00400/) (2023)
- [V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception](https://mlsanity.com/paper/2411.10962/) (2024)
- [GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2511.18729/) (2025)
- [Learning Cooperative Trajectory Representations for Motion Forecasting](https://mlsanity.com/paper/2311.00371/) (2023)
- [HeightFormer: Explicit Height Modeling without Extra Data for Camera-only 3D Object Detection in Bird's Eye View](https://mlsanity.com/paper/2307.13510/) (2023)
- [CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity](https://mlsanity.com/paper/2310.02815/) (2023)
- [AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Model](https://mlsanity.com/paper/2312.13156/) (2023)
- [BEVSpread: Spread Voxel Pooling for Bird's-Eye-View Representation in Vision-based Roadside 3D Object Detection](https://mlsanity.com/paper/2406.08785/) (2024)
- [TraF-Align: Trajectory-aware Feature Alignment for Asynchronous Multi-agent Perception](https://mlsanity.com/paper/2503.19391/) (2025)
- [CARLA Drone: Monocular 3D Object Detection from a Different Perspective](https://mlsanity.com/paper/2408.11958/) (2024)
- [CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting](https://mlsanity.com/paper/2507.18473/) (2025)
- [RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception](https://mlsanity.com/paper/2405.09883/) (2024)
- [WARM-3D: A Weakly-Supervised Sim2Real Domain Adaptation Framework for Roadside Monocular 3D Object Detection](https://mlsanity.com/paper/2407.20818/) (2024)
- [Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism](https://mlsanity.com/paper/2510.19618/) (2025)
- [RoadBEV: Road Surface Reconstruction in Bird's Eye View](https://mlsanity.com/paper/2404.06605/) (2024)
- [Collaborative Perception Datasets for Autonomous Driving: A Review](https://mlsanity.com/paper/2504.12696/) (2025)
- [Scaling Multi-Camera 3D Object Detection through Weak-to-Strong Eliciting](https://mlsanity.com/paper/2404.06700/) (2024)
- [SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2406.10907/) (2024)
- [RopeBEV: A Multi-Camera Roadside Perception Network in Bird's-Eye-View](https://mlsanity.com/paper/2409.11706/) (2024)
- [HeightLane: BEV Heightmap guided 3D Lane Detection](https://mlsanity.com/paper/2408.08270/) (2024)

## Abstract (arXiv)

While most recent autonomous driving system focuses on developing perception methods on ego-vehicle sensors, people tend to overlook an alternative approach to leverage intelligent roadside cameras to extend the perception ability beyond the visual range. We discover that the state-of-the-art vision-centric bird's eye view detection methods have inferior performances on roadside cameras. This is because these methods mainly focus on recovering the depth regarding the camera center, where the depth difference between the car and the ground quickly shrinks while the distance increases. In this paper, we propose a simple yet effective approach, dubbed BEVHeight, to address this issue. In essence, instead of predicting the pixel-wise depth, we regress the height to the ground to achieve a distance-agnostic formulation to ease the optimization process of camera-only perception methods. On popular 3D detection benchmarks of roadside cameras, our method surpasses all previous vision-centric methods by a significant margin. The code is available at urlhttps://github.com/ADLab-AutoDrive/BEVHeight.