# No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles using Cameras & LiDARs

arXiv: https://arxiv.org/abs/1802.08755 · HTML: https://mlsanity.com/paper/1802.08755/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2018.

## Built on by (most cited first, 19 of 19)

- [DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion](https://mlsanity.com/paper/2111.14690/) (2021)
- [Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Sets](https://mlsanity.com/paper/2402.07320/) (2024)
- [TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking](https://mlsanity.com/paper/2101.04206/) (2021)
- [Large Scale Real-World Multi-Person Tracking](https://mlsanity.com/paper/2211.02175/) (2022)
- [On Salience-Sensitive Sign Classification in Autonomous Vehicle Path Planning: Experimental Explorations with a Novel Dataset](https://mlsanity.com/paper/2112.00942/) (2021)
- [Multi-modal Fusion Technology based on Vehicle Information: A Survey](https://mlsanity.com/paper/2211.06080/) (2022)
- [TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos](https://mlsanity.com/paper/2404.13868/) (2024)
- [CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking](https://mlsanity.com/paper/2107.05150/) (2021)
- [Next Generation Multitarget Trackers: Random Finite Set Methods vs Transformer-based Deep Learning](https://mlsanity.com/paper/2104.00734/) (2021)
- [Can Deep Learning be Applied to Model-Based Multi-Object Tracking?](https://mlsanity.com/paper/2202.07909/) (2022)
- [FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2308.07104/) (2023)
- [TrafficMOT: A Challenging Dataset for Multi-Object Tracking in Complex Traffic Scenarios](https://mlsanity.com/paper/2311.18839/) (2023)
- [Image Segmentation to Identify Safe Landing Zones for Unmanned Aerial Vehicles](https://mlsanity.com/paper/2111.14557/) (2021)
- [Privacy-preserving Pedestrian Tracking using Distributed 3D LiDARs](https://mlsanity.com/paper/2303.09915/) (2023)
- [Transformer-Based Multi-Object Smoothing with Decoupled Data Association and Smoothing](https://mlsanity.com/paper/2312.17261/) (2023)
- [AViTMP: A Tracking-Specific Transformer for Single-Branch Visual Tracking](https://mlsanity.com/paper/2310.19542/) (2023)
- [Dynamic and Static Object Detection Considering Fusion Regions and Point-wise Features](https://mlsanity.com/paper/2107.12692/) (2021)
- [Structure Aware and Class Balanced 3D Object Detection on nuScenes Dataset](https://mlsanity.com/paper/2205.12519/) (2022)
- [DMSORT: An efficient parallel maritime multi-object tracking architecture for unmanned vessel platforms](https://mlsanity.com/paper/2511.04128/) (2025)

## Abstract (arXiv)

Online multi-object tracking (MOT) is extremely important for high-level spatial reasoning and path planning for autonomous and highly-automated vehicles. In this paper, we present a modular framework for tracking multiple objects (vehicles), capable of accepting object proposals from different sensor modalities (vision and range) and a variable number of sensors, to produce continuous object tracks. This work is a generalization of the MDP framework for MOT, with some key extensions - First, we track objects across multiple cameras and across different sensor modalities. This is done by fusing object proposals across sensors accurately and efficiently. Second, the objects of interest (targets) are tracked directly in the real world. This is a departure from traditional techniques where objects are simply tracked in the image plane. Doing so allows the tracks to be readily used by an autonomous agent for navigation and related tasks. To verify the effectiveness of our approach, we test it on real world highway data collected from a heavily sensorized testbed capable of capturing full-surround information. We demonstrate that our framework is well-suited to track objects through entire maneuvers around the ego-vehicle, some of which take more than a few minutes to complete. We also leverage the modularity of our approach by comparing the effects of including/excluding different sensors, changing the total number of sensors, and the quality of object proposals on the final tracking result.