# Shift R-CNN: Deep Monocular 3D Object Detection with Closed-Form Geometric Constraints

arXiv: https://arxiv.org/abs/1905.09970 · HTML: https://mlsanity.com/paper/1905.09970/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 25 of 25)

- [Is Pseudo-Lidar needed for Monocular 3D Object detection?](https://mlsanity.com/paper/2108.06417/) (2021)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.16470/) (2021)
- [3D Object Detection from Images for Autonomous Driving: A Survey](https://mlsanity.com/paper/2202.02980/) (2022)
- [Ground-aware Monocular 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2102.00690/) (2021)
- [M3DSSD: Monocular 3D Single Stage Object Detector](https://mlsanity.com/paper/2103.13164/) (2021)
- [Homography Loss for Monocular 3D Object Detection](https://mlsanity.com/paper/2204.00754/) (2022)
- [OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection](https://mlsanity.com/paper/2211.01142/) (2022)
- [MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors](https://mlsanity.com/paper/2410.19590/) (2024)
- [DPFT: Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection](https://mlsanity.com/paper/2404.03015/) (2024)
- [MonoCInIS: Camera Independent Monocular 3D Object Detection using Instance Segmentation](https://mlsanity.com/paper/2110.00464/) (2021)
- [IAFA: Instance-aware Feature Aggregation for 3D Object Detection from a Single Image](https://mlsanity.com/paper/2103.03480/) (2021)
- [Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR](https://mlsanity.com/paper/2109.09628/) (2021)
- [Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Data](https://mlsanity.com/paper/2110.10966/) (2021)
- [3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection](https://mlsanity.com/paper/2507.23567/) (2025)
- [OBMO: One Bounding Box Multiple Objects for Monocular 3D Object Detection](https://mlsanity.com/paper/2212.10049/) (2022)
- [Every Dataset Counts: Scaling up Monocular 3D Object Detection with Joint Datasets Training](https://mlsanity.com/paper/2310.00920/) (2023)
- [Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth](https://mlsanity.com/paper/2107.13269/) (2021)
- [SM3D: Simultaneous Monocular Mapping and 3D Detection](https://mlsanity.com/paper/2111.12643/) (2021)
- [Survey and Systematization of 3D Object Detection Models and Methods](https://mlsanity.com/paper/2201.09354/) (2022)
- [HeightFormer: Learning Height Prediction in Voxel Features for Roadside Vision Centric 3D Object Detection via Transformer](https://mlsanity.com/paper/2503.10777/) (2025)
- [SoK: Vehicle Orientation Representations for Deep Rotation Estimation](https://mlsanity.com/paper/2112.04421/) (2021)
- [Polygon Intersection-over-Union Loss for Viewpoint-Agnostic Monocular 3D Vehicle Detection](https://mlsanity.com/paper/2309.07104/) (2023)
- [Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs](https://mlsanity.com/paper/2608.12179/) (2026)
- [MDS-Net: A Multi-scale Depth Stratification Based Monocular 3D Object Detection Algorithm](https://mlsanity.com/paper/2201.04341/) (2022)

## Abstract (arXiv)

We propose Shift R-CNN, a hybrid model for monocular 3D object detection, which combines deep learning with the power of geometry. We adapt a Faster R-CNN network for regressing initial 2D and 3D object properties and combine it with a least squares solution for the inverse 2D to 3D geometric mapping problem, using the camera projection matrix. The closed-form solution of the mathematical system, along with the initial output of the adapted Faster R-CNN are then passed through a final ShiftNet network that refines the result using our newly proposed Volume Displacement Loss. Our novel, geometrically constrained deep learning approach to monocular 3D object detection obtains top results on KITTI 3D Object Detection Benchmark, being the best among all monocular methods that do not use any pre-trained network for depth estimation.