# Monocular Differentiable Rendering for Self-Supervised 3D Object Detection

arXiv: https://arxiv.org/abs/2009.14524 · HTML: https://mlsanity.com/paper/2009.14524/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 23 of 23)

- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator](https://mlsanity.com/paper/2102.12145/) (2021)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.17202/) (2021)
- [3D Object Detection from Images for Autonomous Driving: A Survey](https://mlsanity.com/paper/2202.02980/) (2022)
- [Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation](https://mlsanity.com/paper/2203.10339/) (2022)
- [3D Object Detection with a Self-supervised Lidar Scene Flow Backbone](https://mlsanity.com/paper/2205.00705/) (2022)
- [Neural Groundplans: Persistent Neural Scene Representations from a Single Image](https://mlsanity.com/paper/2207.11232/) (2022)
- [Rasterized Edge Gradients: Handling Discontinuities Differentiably](https://mlsanity.com/paper/2405.02508/) (2024)
- [NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localization](https://mlsanity.com/paper/2305.17763/) (2023)
- [Detection Based Part-level Articulated Object Reconstruction from Single RGBD Image](https://mlsanity.com/paper/2504.03177/) (2025)
- [Stereo CenterNet based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2103.11071/) (2021)
- [Baking in the Feature: Accelerating Volumetric Segmentation by Rendering Feature Maps](https://mlsanity.com/paper/2209.12744/) (2022)
- [OBMO: One Bounding Box Multiple Objects for Monocular 3D Object Detection](https://mlsanity.com/paper/2212.10049/) (2022)
- [Physics and semantic informed multi-sensor calibration via optimization theory and self-supervised learning](https://mlsanity.com/paper/2206.02856/) (2022)
- [Time-to-Label: Temporal Consistency for Self-Supervised Monocular 3D Object Detection](https://mlsanity.com/paper/2203.02193/) (2022)
- [Category-Level Pose Retrieval with Contrastive Features Learnt with Occlusion Augmentation](https://mlsanity.com/paper/2208.06195/) (2022)
- [M&M3D: Multi-Dataset Training and Efficient Network for Multi-view 3D Object Detection](https://mlsanity.com/paper/2311.00986/) (2023)
- [Inverse Neural Rendering for Explainable Multi-Object Tracking](https://mlsanity.com/paper/2404.12359/) (2024)
- [MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos](https://mlsanity.com/paper/2601.02716/) (2026)
- [Self-Supervised 3D Monocular Object Detection by Recycling Bounding Boxes](https://mlsanity.com/paper/2206.12738/) (2022)
- [View-to-Label: Multi-View Consistency for Self-Supervised 3D Object Detection](https://mlsanity.com/paper/2305.17972/) (2023)

## Abstract (arXiv)

3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale. To overcome this ambiguity, we present a novel self-supervised method for textured 3D shape reconstruction and pose estimation of rigid objects with the help of strong shape priors and 2D instance masks. Our method predicts the 3D location and meshes of each object in an image using differentiable rendering and a self-supervised objective derived from a pretrained monocular depth estimation network. We use the KITTI 3D object detection dataset to evaluate the accuracy of the method. Experiments demonstrate that we can effectively use noisy monocular depth and differentiable rendering as an alternative to expensive 3D ground-truth labels or LiDAR information.