# Monocular 3D Object Detection and Box Fitting Trained End-to-End Using Intersection-over-Union Loss

arXiv: https://arxiv.org/abs/1906.08070 · HTML: https://mlsanity.com/paper/1906.08070/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2019.

## Built on by (most cited first, 30 of 30)

- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [Is Pseudo-Lidar needed for Monocular 3D Object detection?](https://mlsanity.com/paper/2108.06417/) (2021)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Probabilistic and Geometric Depth: Detecting Objects in Perspective](https://mlsanity.com/paper/2107.14160/) (2021)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation](https://mlsanity.com/paper/2204.05088/) (2022)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.16470/) (2021)
- [Geometry-based Distance Decomposition for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.03775/) (2021)
- [Ground-aware Monocular 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2102.00690/) (2021)
- [Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2211.07171/) (2022)
- [M3DSSD: Monocular 3D Single Stage Object Detector](https://mlsanity.com/paper/2103.13164/) (2021)
- [Monocular 3D Object Detection with Depth from Motion](https://mlsanity.com/paper/2207.12988/) (2022)
- [Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview](https://mlsanity.com/paper/2105.14291/) (2021)
- [MonoGRNet: A General Framework for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.08797/) (2021)
- [Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection](https://mlsanity.com/paper/2107.13931/) (2021)
- [Weakly Supervised Monocular 3D Object Detection using Multi-View Projection and Direction Consistency](https://mlsanity.com/paper/2303.08686/) (2023)
- [MonoCInIS: Camera Independent Monocular 3D Object Detection using Instance Segmentation](https://mlsanity.com/paper/2110.00464/) (2021)
- [IAFA: Instance-aware Feature Aggregation for 3D Object Detection from a Single Image](https://mlsanity.com/paper/2103.03480/) (2021)
- [Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR](https://mlsanity.com/paper/2109.09628/) (2021)
- [Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Data](https://mlsanity.com/paper/2110.10966/) (2021)
- [Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking](https://mlsanity.com/paper/2206.03666/) (2022)
- [Every Dataset Counts: Scaling up Monocular 3D Object Detection with Joint Datasets Training](https://mlsanity.com/paper/2310.00920/) (2023)
- [Survey and Systematization of 3D Object Detection Models and Methods](https://mlsanity.com/paper/2201.09354/) (2022)
- [PersDet: Monocular 3D Detection in Perspective Bird's-Eye-View](https://mlsanity.com/paper/2208.09394/) (2022)
- [SoK: Vehicle Orientation Representations for Deep Rotation Estimation](https://mlsanity.com/paper/2112.04421/) (2021)
- [Scalable Vision-Based 3D Object Detection and Monocular Depth Estimation for Autonomous Driving](https://mlsanity.com/paper/2403.02037/) (2024)

## Abstract (arXiv)

Three-dimensional object detection from a single view is a challenging task which, if performed with good accuracy, is an important enabler of low-cost mobile robot perception. Previous approaches to this problem suffer either from an overly complex inference engine or from an insufficient detection accuracy. To deal with these issues, we present SS3D, a single-stage monocular 3D object detector. The framework consists of (i) a CNN, which outputs a redundant representation of each relevant object in the image with corresponding uncertainty estimates, and (ii) a 3D bounding box optimizer. We show how modeling heteroscedastic uncertainty improves performance upon our baseline, and furthermore, how back-propagation can be done through the optimizer in order to train the pipeline end-to-end for additional accuracy. Our method achieves SOTA accuracy on monocular 3D object detection, while running at 20 fps in a straightforward implementation. We argue that the SS3D architecture provides a solid framework upon which high performing detection systems can be built, with autonomous driving being the main application in mind.