# Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing

arXiv: https://arxiv.org/abs/2310.08114 · HTML: https://mlsanity.com/paper/2310.08114/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2023.

## Built on by (most cited first, 4 of 4)

- [Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors](https://mlsanity.com/paper/2407.08049/) (2024)
- [OptiPMB: Enhancing 3D Multi-Object Tracking with Optimized Poisson Multi-Bernoulli Filtering](https://mlsanity.com/paper/2503.12968/) (2025)
- [HiLO: High-Level Object Fusion for Autonomous Driving using Transformers](https://mlsanity.com/paper/2506.02554/) (2025)
- [A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing](https://mlsanity.com/paper/2609.08338/) (2026)

## Abstract (arXiv)

Reliable detection and tracking of surrounding objects are indispensable for comprehensive motion prediction and planning of autonomous vehicles. Due to the limitations of individual sensors, the fusion of multiple sensor modalities is required to improve the overall detection capabilities. Additionally, robust motion tracking is essential for reducing the effect of sensor noise and improving state estimation accuracy. The reliability of the autonomous vehicle software becomes even more relevant in complex, adversarial high-speed scenarios at the vehicle handling limits in autonomous racing. In this paper, we present a modular multi-modal sensor fusion and tracking method for high-speed applications. The method is based on the Extended Kalman Filter (EKF) and is capable of fusing heterogeneous detection inputs to track surrounding objects consistently. A novel delay compensation approach enables to reduce the influence of the perception software latency and to output an updated object list. It is the first fusion and tracking method validated in high-speed real-world scenarios at the Indy Autonomous Challenge 2021 and the Autonomous Challenge at CES (AC@CES) 2022, proving its robustness and computational efficiency on embedded systems. It does not require any labeled data and achieves position tracking residuals below 0.1 m. The related code is available as open-source software at https://github.com/TUMFTM/FusionTracking.