# EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View

arXiv: https://arxiv.org/abs/2310.13350 · HTML: https://mlsanity.com/paper/2310.13350/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?](https://mlsanity.com/paper/2206.07959/) (2022)
- [CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification](https://mlsanity.com/paper/1903.09254/) (2019)
- [Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification](https://mlsanity.com/paper/1809.04427/) (2018)
- [Multiview Detection with Feature Perspective Transformation](https://mlsanity.com/paper/2007.07247/) (2020)
- [Multiview Detection with Shadow Transformer (and View-Coherent Data Augmentation)](https://mlsanity.com/paper/2108.05888/) (2021)
- [Simple Cues Lead to a Strong Multi-Object Tracker](https://mlsanity.com/paper/2206.04656/) (2022)
- [Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection](https://mlsanity.com/paper/1704.05775/) (2017)
- [Deep Multi-camera People Detection](https://mlsanity.com/paper/1702.04593/) (2017)
- [LMGP: Lifted Multicut Meets Geometry Projections for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2111.11892/) (2021)
- [ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2308.13229/) (2023)
- [Real-time 3D Deep Multi-Camera Tracking](https://mlsanity.com/paper/2003.11753/) (2020)
- [3D Random Occlusion and Multi-Layer Projection for Deep Multi-Camera Pedestrian Localization](https://mlsanity.com/paper/2207.10895/) (2022)
- [Lightweight Multi-Branch Network for Person Re-Identification](https://mlsanity.com/paper/2101.10774/) (2021)
- [Synthehicle: Multi-Vehicle Multi-Camera Tracking in Virtual Cities](https://mlsanity.com/paper/2208.14167/) (2022)

## Built on by (most cited first, 30 of 33)

- [Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perception](https://mlsanity.com/paper/2403.07746/) (2024)
- [Lifting Multi-View Detection and Tracking to the Bird's Eye View](https://mlsanity.com/paper/2403.12573/) (2024)
- [MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos](https://mlsanity.com/paper/2412.00692/) (2024)
- [SpaRC: Sparse Radar-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2411.19860/) (2024)
- [Enhanced Multi-View Pedestrian Detection Using Probabilistic Occupancy Volume](https://mlsanity.com/paper/2503.10982/) (2025)
- [MITracker: Multi-View Integration for Visual Object Tracking](https://mlsanity.com/paper/2502.20111/) (2025)
- [DCHM: Depth-Consistent Human Modeling for Multiview Detection](https://mlsanity.com/paper/2507.14505/) (2025)
- [MVTrajecter: Multi-View Pedestrian Tracking with Trajectory Motion Cost and Trajectory Appearance Cost](https://mlsanity.com/paper/2509.01157/) (2025)
- [Resource-Efficient Multiview Perception: Integrating Semantic Masking with Masked Autoencoders](https://mlsanity.com/paper/2410.04817/) (2024)
- [3D Multi-Object Tracking Employing MS-GLMB Filter for Autonomous Driving](https://mlsanity.com/paper/2410.14977/) (2024)
- [MVUDA: Unsupervised Domain Adaptation for Multi-view Pedestrian Detection](https://mlsanity.com/paper/2412.04117/) (2024)
- [Attention-Aware Multi-View Pedestrian Tracking](https://mlsanity.com/paper/2504.03047/) (2025)
- [One Graph to Track Them All: Dynamic GNNs for Single- and Multi-View Tracking](https://mlsanity.com/paper/2507.08494/) (2025)
- [Online 3D Multi-Camera Perception through Robust 2D Tracking and Depth-based Late Aggregation](https://mlsanity.com/paper/2509.09946/) (2025)
- [A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systems](https://mlsanity.com/paper/2601.10819/) (2026)
- [Spatial-Temporal Multi-Cuts for Online Multiple-Camera Vehicle Tracking](https://mlsanity.com/paper/2410.02638/) (2024)
- [SpaRC-AD: A Baseline for Radar-Camera Fusion in End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.10567/) (2025)
- [ATLASFusion: Aggregation Tracking with Location-Aware Sparse Fusion for Robust Spatio-Temporal Multi-View Pedestrian Tracking](https://mlsanity.com/paper/2509.08421/) (2025)
- [Multi Camera Connected Vision System with Multi View Analytics: A Comprehensive Survey](https://mlsanity.com/paper/2510.09731/) (2025)
- [3D Ground Truth Reconstruction from Multi-Camera Annotations Using UKF](https://mlsanity.com/paper/2511.17609/) (2025)
- [Model Optimization for Multi-Camera 3D Detection and Tracking](https://mlsanity.com/paper/2602.00450/) (2026)
- [ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation](https://mlsanity.com/paper/2603.15812/) (2026)
- [LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving](https://mlsanity.com/paper/2603.21987/) (2026)
- [SynMVCrowd: A Large Synthetic Benchmark for Multi-view Crowd Counting and Localization](https://mlsanity.com/paper/2603.23956/) (2026)
- [Efficient Online 3D Multi-Camera Multi-Object Tracking and Pose Estimation](https://mlsanity.com/paper/2604.16522/) (2026)
- [Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes](https://mlsanity.com/paper/2604.19318/) (2026)
- [ARGUSTRACK: A Multi-View Annotation System for Multi-Object Tracking](https://mlsanity.com/paper/2606.20687/) (2026)
- [MVDGC: Joint 3D and 2D Multi-view Pedestrian Detection via Dual Geometric Constraints](https://mlsanity.com/paper/2607.00273/) (2026)
- [Syn2RealTrack: Bridging the Gap Between Synthetic and Real-World Datasets for Online Multi-View Multi-Target Tracking](https://mlsanity.com/paper/2608.24130/) (2026)
- [GRACE: Geometry- and Ray-Aware Camera-Efficient Multi-View Pedestrian Tracking](https://mlsanity.com/paper/2609.16872/) (2026)

## Abstract (arXiv)

Multi-view aggregation promises to overcome the occlusion and missed detection challenge in multi-object detection and tracking. Recent approaches in multi-view detection and 3D object detection made a huge performance leap by projecting all views to the ground plane and performing the detection in the Bird's Eye View (BEV). In this paper, we investigate if tracking in the BEV can also bring the next performance breakthrough in Multi-Target Multi-Camera (MTMC) tracking. Most current approaches in multi-view tracking perform the detection and tracking task in each view and use graph-based approaches to perform the association of the pedestrian across each view. This spatial association is already solved by detecting each pedestrian once in the BEV, leaving only the problem of temporal association. For the temporal association, we show how to learn strong Re-Identification (re-ID) features for each detection. The results show that early-fusion in the BEV achieves high accuracy for both detection and tracking. EarlyBird outperforms the state-of-the-art methods and improves the current state-of-the-art on Wildtrack by +4.6 MOTA and +5.6 IDF1.