# Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Data

arXiv: https://arxiv.org/abs/2110.10966 · HTML: https://mlsanity.com/paper/2110.10966/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Disentangling Monocular 3D Object Detection](https://mlsanity.com/paper/1905.12365/) (2019)
- [SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation](https://mlsanity.com/paper/2002.10111/) (2020)
- [Learning Depth-Guided Convolutions for Monocular 3D Object Detection](https://mlsanity.com/paper/1912.04799/) (2019)
- [RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving](https://mlsanity.com/paper/2001.03343/) (2020)
- [GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving](https://mlsanity.com/paper/1903.10955/) (2019)
- [Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image](https://mlsanity.com/paper/1703.07570/) (2017)
- [Learning Monocular 3D Human Pose Estimation from Multi-view Images](https://mlsanity.com/paper/1803.04775/) (2018)
- [Deep Fitting Degree Scoring Network for Monocular 3D Object Detection](https://mlsanity.com/paper/1904.12681/) (2019)
- [Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild](https://mlsanity.com/paper/2003.07581/) (2020)
- [Monocular 3D Object Detection via Geometric Reasoning on Keypoints](https://mlsanity.com/paper/1905.05618/) (2019)
- [Monocular 3D Object Detection and Box Fitting Trained End-to-End Using Intersection-over-Union Loss](https://mlsanity.com/paper/1906.08070/) (2019)
- [Shift R-CNN: Deep Monocular 3D Object Detection with Closed-Form Geometric Constraints](https://mlsanity.com/paper/1905.09970/) (2019)
- [MonoGRNet: A General Framework for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.08797/) (2021)

## Built on by (most cited first, 11 of 11)

- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting](https://mlsanity.com/paper/2305.05938/) (2023)
- [Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges](https://mlsanity.com/paper/2301.06262/) (2023)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)
- [DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception](https://mlsanity.com/paper/2310.08117/) (2023)
- [Collaborative Perception Datasets for Autonomous Driving: A Review](https://mlsanity.com/paper/2504.12696/) (2025)
- [Collective Perception Datasets for Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2405.16973/) (2024)
- [Communication-Critical Planning via Multi-Agent Trajectory Exchange](https://mlsanity.com/paper/2303.06080/) (2023)
- [InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios](https://mlsanity.com/paper/2407.21581/) (2024)
- [Cooperative Visual-LiDAR Extrinsic Calibration Technology for Intersection Vehicle-Infrastructure: A review](https://mlsanity.com/paper/2405.10132/) (2024)

## Abstract (arXiv)

Accurate 7DoF prediction of vehicles at an intersection is an important task for assessing potential conflicts between road users. In principle, this could be achieved by a single camera system that is capable of detecting the pose of each vehicle but this would require a large, accurately labelled dataset from which to train the detector. Although large vehicle pose datasets exist (ostensibly developed for autonomous vehicles), we find training on these datasets inadequate. These datasets contain images from a ground level viewpoint, whereas an ideal view for intersection observation would be elevated higher above the road surface. We develop an alternative approach using a weakly supervised method of fine tuning 3D object detectors for traffic observation cameras; showing in the process that large existing autonomous vehicle datasets can be leveraged for pre-training. To fine-tune the monocular 3D object detector, our method utilises multiple 2D detections from overlapping, wide-baseline views and a loss that encodes the subjacent geometric consistency. Our method achieves vehicle 7DoF pose prediction accuracy on our dataset comparable to the top performing monocular 3D object detectors on autonomous vehicle datasets. We present our training methodology, multi-view reprojection loss, and dataset.