# Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Tracking

arXiv: https://arxiv.org/abs/2603.08199 · HTML: https://mlsanity.com/paper/2603.08199/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Builds on (most cited first, 25 of 25)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Focal Sparse Convolutional Networks for 3D Object Detection](https://mlsanity.com/paper/2204.12463/) (2022)
- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries](https://mlsanity.com/paper/2205.00613/) (2022)
- [EagerMOT: 3D Multi-Object Tracking via Sensor Fusion](https://mlsanity.com/paper/2104.14682/) (2021)
- [LargeKernel3D: Scaling up Kernels in 3D Sparse CNNs](https://mlsanity.com/paper/2206.10555/) (2022)
- [CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion](https://mlsanity.com/paper/2209.02540/) (2022)
- [DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association](https://mlsanity.com/paper/2202.12100/) (2022)
- [ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association](https://mlsanity.com/paper/2405.08909/) (2024)
- [Score refinement for confidence-based 3D multi-object tracking](https://mlsanity.com/paper/2107.04327/) (2021)
- [Poly-MOT: A Polyhedral Framework For 3D Multi-Object Tracking](https://mlsanity.com/paper/2307.16675/) (2023)
- [MV2DFusion: Leveraging Modality-Specific Object Semantics for Multi-Modal 3D Detection](https://mlsanity.com/paper/2408.05945/) (2024)
- [MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion](https://mlsanity.com/paper/2306.17000/) (2023)
- [MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving](https://mlsanity.com/paper/2409.16149/) (2024)
- [S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking with Adaptive Spatio-Temporal Appearance Representations](https://mlsanity.com/paper/2306.17602/) (2023)
- [Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking](https://mlsanity.com/paper/2510.10287/) (2025)
- [TQD-Track: Temporal Query Denoising for 3D Multi-Object Tracking](https://mlsanity.com/paper/2504.03258/) (2025)
- [Rethinking the Spatio-Temporal Alignment of End-to-End 3D Perception](https://mlsanity.com/paper/2512.23635/) (2025)
- [SynCL: A Synergistic Training Strategy with Instance-Aware Contrastive Learning for End-to-End Multi-Camera 3D Tracking](https://mlsanity.com/paper/2411.06780/) (2024)
- [Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Tracking](https://mlsanity.com/paper/2602.13772/) (2026)

## Built on by (most cited first, 1 of 1)

- [MatchFusion: Explicit-Implicit Instance Matching for Spatio-Temporal Multimodal Autonomous Driving](https://mlsanity.com/paper/2609.25860/) (2026)

## Abstract (arXiv)

LiDAR-camera 3D multi-object tracking (MOT) combines rich visual semantics with accurate depth cues to improve trajectory consistency and tracking reliability. In practice, however, LiDAR and cameras operate at different sampling rates. To maintain temporal alignment, existing data pipelines usually synchronize heterogeneous sensor streams and annotate them at a reduced shared frequency, forcing most prior methods to perform spatial fusion only at synchronized timestamps through projection-based or learnable cross-sensor association. As a result, abundant asynchronous observations remain underexploited, despite their potential to support more frequent association and more robust trajectory estimation over short temporal intervals. To address this limitation, we propose Fusion-Poly, a spatial-temporal fusion framework for 3D MOT that integrates asynchronous LiDAR and camera data. Fusion-Poly associates trajectories with multi-modal observations at synchronized timestamps and with single-modal observations at asynchronous timestamps, enabling higher-frequency updates of motion and existence states. The framework contains three key components: a frequency-aware cascade matching module that adapts to synchronized and asynchronous frames according to available detection modalities; a frequency-aware trajectory estimation module that maintains trajectories through high-frequency motion prediction, differential updates, and confidence-calibrated lifecycle management; and a full-state observation alignment module that improves cross-modal consistency at synchronized timestamps by optimizing image-projection errors. On the nuScenes test set, Fusion-Poly achieves 76.5% AMOTA, establishing a new state of the art among tracking-by-detection 3D MOT methods. Extensive ablation studies further validate the effectiveness of each component. Code will be released.