# Mono-Camera 3D Multi-Object Tracking Using Deep Learning Detections and PMBM Filtering arXiv: https://arxiv.org/abs/1802.09975 ยท HTML: https://mlsanity.com/paper/1802.09975/ ## Key facts - Cited by 32 papers in the map (data through 14 Oct 2026). - First posted on arXiv 27 Feb 2018. ## Built on by (most cited first, 30 of 32) - [ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries](https://mlsanity.com/paper/2208.01582/) (2022) - [MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries](https://mlsanity.com/paper/2205.00613/) (2022) - [Monocular Quasi-Dense 3D Object Tracking](https://mlsanity.com/paper/2103.07351/) (2021) - [3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds](https://mlsanity.com/paper/2111.04426/) (2021) - [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023) - [CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion](https://mlsanity.com/paper/2209.02540/) (2022) - [3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud](https://mlsanity.com/paper/2108.05630/) (2021) - [3D Siamese Transformer Network for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2207.11995/) (2022) - [Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving](https://mlsanity.com/paper/2108.04602/) (2021) - [TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM](https://mlsanity.com/paper/2210.16204/) (2022) - [A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds](https://mlsanity.com/paper/2203.04232/) (2022) - [TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking](https://mlsanity.com/paper/2101.04206/) (2021) - [MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion](https://mlsanity.com/paper/2306.17000/) (2023) - [MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation](https://mlsanity.com/paper/2110.09481/) (2021) - [Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review](https://mlsanity.com/paper/2203.10939/) (2022) - [Neural Enhanced Belief Propagation for Multiobject Tracking](https://mlsanity.com/paper/2212.08340/) (2022) - [Neural Enhanced Belief Propagation for Data Association in Multiobject Tracking](https://mlsanity.com/paper/2203.09948/) (2022) - [PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search](https://mlsanity.com/paper/2403.15712/) (2024) - [Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration](https://mlsanity.com/paper/2105.14683/) (2021) - [Depth Perspective-aware Multiple Object Tracking](https://mlsanity.com/paper/2207.04551/) (2022) - [InterTrack: Interaction Transformer for 3D Multi-Object Tracking](https://mlsanity.com/paper/2208.08041/) (2022) - [Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking](https://mlsanity.com/paper/2203.16268/) (2022) - [Pedestrian Tracking with Monocular Camera using Unconstrained 3D Motion Model](https://mlsanity.com/paper/2403.11978/) (2024) - [A two-stage data association approach for 3D Multi-object Tracking](https://mlsanity.com/paper/2101.08684/) (2021) - [HybridTrack: A Hybrid Approach for Robust Multi-Object Tracking](https://mlsanity.com/paper/2501.01275/) (2025) - [Trajectory Poisson multi-Bernoulli mixture filter for traffic monitoring using a drone](https://mlsanity.com/paper/2306.16890/) (2023) - [An End-to-End Framework of Road User Detection, Tracking, and Prediction from Monocular Images](https://mlsanity.com/paper/2308.05026/) (2023) - [Inverse Neural Rendering for Explainable Multi-Object Tracking](https://mlsanity.com/paper/2404.12359/) (2024) - [DeepKalPose: An Enhanced Deep-Learning Kalman Filter for Temporally Consistent Monocular Vehicle Pose Estimation](https://mlsanity.com/paper/2404.16558/) (2024) - [OptiPMB: Enhancing 3D Multi-Object Tracking with Optimized Poisson Multi-Bernoulli Filtering](https://mlsanity.com/paper/2503.12968/) (2025) ## Abstract (arXiv) Monocular cameras are one of the most commonly used sensors in the automotive industry for autonomous vehicles. One major drawback using a monocular camera is that it only makes observations in the two dimensional image plane and can not directly measure the distance to objects. In this paper, we aim at filling this gap by developing a multi-object tracking algorithm that takes an image as input and produces trajectories of detected objects in a world coordinate system. We solve this by using a deep neural network trained to detect and estimate the distance to objects from a single input image. The detections from a sequence of images are fed in to a state-of-the art Poisson multi-Bernoulli mixture tracking filter. The combination of the learned detector and the PMBM filter results in an algorithm that achieves 3D tracking using only mono-camera images as input. The performance of the algorithm is evaluated both in 3D world coordinates, and 2D image coordinates, using the publicly available KITTI object tracking dataset. The algorithm shows the ability to accurately track objects, correctly handle data associations, even when there is a big overlap of the objects in the image, and is one of the top performing algorithms on the KITTI object tracking benchmark. Furthermore, the algorithm is efficient, running on average close to 20 frames per second.