# A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing

arXiv: https://arxiv.org/abs/2609.08338 · HTML: https://mlsanity.com/paper/2609.08338/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [RACECAR -- The Dataset for High-Speed Autonomous Racing](https://mlsanity.com/paper/2306.03252/) (2023)
- [Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing](https://mlsanity.com/paper/2310.08114/) (2023)
- [MonoDINO-DETR: Depth-Enhanced Monocular 3D Object Detection Using a Vision Foundation Model](https://mlsanity.com/paper/2502.00315/) (2025)
- [Fast and Modular Autonomy Software for Autonomous Racing Vehicles](https://mlsanity.com/paper/2408.15425/) (2024)
- [BETTY Dataset: A Multi-modal Dataset for Full-Stack Autonomy](https://mlsanity.com/paper/2505.07266/) (2025)
- [LiDAR-Based Vehicle Detection and Tracking for Autonomous Racing](https://mlsanity.com/paper/2501.14502/) (2025)
- [A Racing Dataset and Baseline Model for Track Detection in Autonomous Racing](https://mlsanity.com/paper/2502.14068/) (2025)

## Abstract (arXiv)

Object detection and tracking are fundamental components of perception systems for autonomous driving. Achieving robust performance under adverse conditions such as limited visibility, sensor noise, and failures remains an open challenge, particularly in autonomous racing, where vehicles operate at very high speeds, experience strong vibrations, and interact under small safety margins. This paper presents a multi-modal late-fusion perception pipeline for object detection and tracking in the autonomous racing domain. The proposed system extends previous work by exploiting all onboard sensors through a late-fusion approach and a dedicated multi-object tracking framework. Independent detections from cameras, LiDARs, and RADARs are combined to provide timely and robust state estimates of surrounding vehicles. The tracking method explicitly compensates for detection delays and embeds in its model prior knowledge of vehicle dynamics and track layout. Experimental evaluation on real-world data across diverse critical scenarios, representative of challenging edge cases also in urban driving, confirms the effectiveness of the proposed pipeline and its suitability to support safe and adaptive planning decisions.