mlsanityData to 14 Oct 2026

HeightFormer: Explicit Height Modeling without Extra Data for Camera-only 3D Object Detection in Bird's Eye View

arXiv 2307.13510 ↗First posted 25 Jul 2023cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Vision-based Bird's Eye View (BEV) representation is an emerging perception formulation for autonomous driving. The core challenge is to construct BEV space with multi-camera features, which is a one-to-many ill-posed problem. Diving into all previous BEV representation generation methods, we found that most of them fall into two types: modeling depths in image views or modeling heights in the BEV space, mostly in an implicit way. In this work, we propose to explicitly model heights in the BEV space, which needs no extra data like LiDAR and can fit arbitrary camera rigs and types compared to modeling depths. Theoretically, we give proof of the equivalence between height-based methods and depth-based methods. Considering the equivalence and some advantages of modeling heights, we propose HeightFormer, which models heights and uncertainties in a self-recursive way. Without any extra data, the proposed HeightFormer could estimate heights in BEV accurately. Benchmark results show that the performance of HeightFormer achieves SOTA compared with those camera-only methods.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 50 of its 57 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

50 of 57 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  9. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  12. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
38 more
  1. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  2. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  3. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  4. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  5. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  6. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  7. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  8. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  9. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  10. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  11. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  12. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 202
  13. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  14. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  15. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  16. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  17. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 185
  18. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  19. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  20. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  21. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  22. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 156
  23. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
  24. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  25. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  26. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  27. 2021Geometry-based Distance Decomposition for Monocular 3D Object Detectioncited by 84
  28. 2023BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detectioncited by 75
  29. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  30. 2020Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimationcited by 43
  31. 2022Learning Ego 3D Representation as Ray Tracingcited by 43
  32. 2023Fast-BEV: A Fast and Strong Bird's-Eye View Perception Baselinecited by 40
  33. 2022MonoGround: Detecting Monocular 3D Objects from the Groundcited by 37
  34. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  35. 2022LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentationcited by 29
  36. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  37. 2021X-view: Non-egocentric Multi-View 3D Object Detectorcited by 7
  38. 2022Monocular BEV Perception of Road Scenes via Front-to-Top View Projectioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 6; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  2. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  3. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 23
  4. 2024BEVSpread: Spread Voxel Pooling for Bird's-Eye-View Representation in Vision-based Roadside 3D Object Detectioncited by 15
  5. 2024DualBEV: Unifying Dual View Transformation with Probabilistic Correspondencescited by 9
  6. 2024FlatFusion: Delving into Details of Sparse Transformer-based Camera-LiDAR Fusion for Autonomous Drivingcited by 9
  7. 2024Anchor3DLane++: 3D Lane Detection via Sample-Adaptive Sparse 3D Anchor Regressioncited by 8
  8. 2025Spatial Retrieval Augmented Autonomous Drivingcited by 5
  9. 2024Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detectioncited by 4
  10. 2024HeightLane: BEV Heightmap guided 3D Lane Detectioncited by 4
  11. 2024HeightMapNet: Explicit Height Modeling for End-to-End HD Map Learningcited by 3
  12. 2024HV-BEV: Decoupling Horizontal and Vertical Feature Sampling for Multi-View 3D Object Detectioncited by 3
7 more
  1. 2025DB3D-L: Depth-aware BEV Feature Transformation for Accurate 3D Lane Detectioncited by 1
  2. 2025OcRFDet: Object-Centric Radiance Fields for Multi-View 3D Object Detection in Autonomous Drivingcited by 1
  3. 2025SC-Lane: Slope-aware and Consistent Road Height Estimation Framework for 3D Lane Detectioncited by 1
  4. 2025MAESTRO: Task-Relevant Optimization via Adaptive Feature Enhancement and Suppression for Multi-task 3D Perceptioncited by 0
  5. 2026Multi-Modal Decouple and Recouple Network for Robust 3D Object Detectioncited by 0
  6. 2026HSDF-Lane: Height-Aligned Signed Distance Field with Semantic Lane Prior for 3D Lane Detectioncited by 0
  7. 2026FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awarenesscited by 0