# Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

arXiv: https://arxiv.org/abs/2112.04628 · HTML: https://mlsanity.com/paper/2112.04628/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 69 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 30 of 57)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [Disentangling Monocular 3D Object Detection](https://mlsanity.com/paper/1905.12365/) (2019)
- [SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation](https://mlsanity.com/paper/2002.10111/) (2020)
- [Probabilistic and Geometric Depth: Detecting Objects in Perspective](https://mlsanity.com/paper/2107.14160/) (2021)
- [Objects are Different: Flexible Monocular 3D Object Detection](https://mlsanity.com/paper/2104.02323/) (2021)
- [Learning Depth-Guided Convolutions for Monocular 3D Object Detection](https://mlsanity.com/paper/1912.04799/) (2019)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [Geometry Uncertainty Projection Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2107.13774/) (2021)
- [MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships](https://mlsanity.com/paper/2003.00504/) (2020)
- [RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving](https://mlsanity.com/paper/2001.03343/) (2020)
- [Delving into Localization Errors for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.16237/) (2021)
- [Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving](https://mlsanity.com/paper/1903.11444/) (2019)
- [ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape](https://mlsanity.com/paper/1812.02781/) (2018)

## Built on by (most cited first, 30 of 67)

- [BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision](https://mlsanity.com/paper/2211.10439/) (2022)
- [Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild](https://mlsanity.com/paper/2207.10660/) (2022)
- [3D Object Detection from Images for Autonomous Driving: A Survey](https://mlsanity.com/paper/2202.02980/) (2022)
- [DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3D Object Detection](https://mlsanity.com/paper/2207.10758/) (2022)
- [CAPE: Camera View Position Embedding for Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.10209/) (2023)
- [DID-M3D: Decoupling Instance Depth for Monocular 3D Object Detection](https://mlsanity.com/paper/2207.08531/) (2022)
- [MonoCD: Monocular 3D Object Detection with Complementary Depths](https://mlsanity.com/paper/2404.03181/) (2024)
- [Learning Occupancy for Monocular 3D Object Detection](https://mlsanity.com/paper/2305.15694/) (2023)
- [OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection](https://mlsanity.com/paper/2211.01142/) (2022)
- [MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors](https://mlsanity.com/paper/2410.19590/) (2024)
- [MonoPGC: Monocular 3D Object Detection with Pixel Geometry Contexts](https://mlsanity.com/paper/2302.10549/) (2023)
- [MonoLSS: Learnable Sample Selection For Monocular 3D Detection](https://mlsanity.com/paper/2312.14474/) (2023)
- [Weakly Supervised Monocular 3D Object Detection using Multi-View Projection and Direction Consistency](https://mlsanity.com/paper/2303.08686/) (2023)
- [MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders](https://mlsanity.com/paper/2405.07696/) (2024)
- [CARLA Drone: Monocular 3D Object Detection from a Different Perspective](https://mlsanity.com/paper/2408.11958/) (2024)
- [Decoupled Pseudo-labeling for Semi-Supervised Monocular 3D Object Detection](https://mlsanity.com/paper/2403.17387/) (2024)
- [MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection](https://mlsanity.com/paper/2404.04910/) (2024)
- [MonoWAD: Weather-Adaptive Diffusion Model for Robust Monocular 3D Object Detection](https://mlsanity.com/paper/2407.16448/) (2024)
- [Unleashing the Power of Chain-of-Prediction for Monocular 3D Object Detection](https://mlsanity.com/paper/2505.04594/) (2025)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [Monocular 3D Object Detection with Bounding Box Denoising in 3D by Perceiver](https://mlsanity.com/paper/2304.01289/) (2023)
- [Depth-discriminative Metric Learning for Monocular 3D Object Detection](https://mlsanity.com/paper/2401.01075/) (2024)
- [Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection](https://mlsanity.com/paper/2211.16779/) (2022)
- [Multi-View Attentive Contextualization for Multi-View 3D Object Detection](https://mlsanity.com/paper/2405.12200/) (2024)
- [CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer](https://mlsanity.com/paper/2406.08152/) (2024)
- [Shape-Aware Monocular 3D Object Detection](https://mlsanity.com/paper/2204.08717/) (2022)
- [Sample-Level Weighting for Multi-Task Learning with Auxiliary Tasks](https://mlsanity.com/paper/2306.04519/) (2023)
- [Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator](https://mlsanity.com/paper/2403.03468/) (2024)
- [OBMO: One Bounding Box Multiple Objects for Monocular 3D Object Detection](https://mlsanity.com/paper/2212.10049/) (2022)
- [ClusterFusion: Leveraging Radar Spatial Features for Radar-Camera 3D Object Detection in Autonomous Vehicles](https://mlsanity.com/paper/2309.03734/) (2023)

## Abstract (arXiv)

Monocular 3D object detection aims to localize 3D bounding boxes in an input single 2D image. It is a highly challenging problem and remains open, especially when no extra information (e.g., depth, lidar and/or multi-frames) can be leveraged in training and/or inference. This paper proposes a simple yet effective formulation for monocular 3D object detection without exploiting any extra information. It presents the MonoCon method which learns Monocular Contexts, as auxiliary tasks in training, to help monocular 3D object detection. The key idea is that with the annotated 3D bounding boxes of objects in an image, there is a rich set of well-posed projected 2D supervision signals available in training, such as the projected corner keypoints and their associated offset vectors with respect to the center of 2D bounding box, which should be exploited as auxiliary tasks in training. The proposed MonoCon is motivated by the Cramer-Wold theorem in measure theory at a high level. In implementation, it utilizes a very simple end-to-end design to justify the effectiveness of learning auxiliary monocular contexts, which consists of three components: a Deep Neural Network (DNN) based feature backbone, a number of regression head branches for learning the essential parameters used in the 3D bounding box prediction, and a number of regression head branches for learning auxiliary contexts. After training, the auxiliary context regression branches are discarded for better inference efficiency. In experiments, the proposed MonoCon is tested in the KITTI benchmark (car, pedestrain and cyclist). It outperforms all prior arts in the leaderboard on car category and obtains comparable performance on pedestrian and cyclist in terms of accuracy. Thanks to the simple design, the proposed MonoCon method obtains the fastest inference speed with 38.7 fps in comparisons