# Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions

arXiv: https://arxiv.org/abs/2007.13971 · HTML: https://mlsanity.com/paper/2007.13971/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 11 of 11)

- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo](https://mlsanity.com/paper/2406.12563/) (2024)
- [Safe Autonomous Racing via Approximate Reachability on Ego-vision](https://mlsanity.com/paper/2110.07699/) (2021)
- [Analyzing Deep Learning Representations of Point Clouds for Real-Time In-Vehicle LiDAR Perception](https://mlsanity.com/paper/2210.14612/) (2022)
- [Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras](https://mlsanity.com/paper/2107.08862/) (2021)
- [Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing](https://mlsanity.com/paper/2205.02953/) (2022)
- [Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing](https://mlsanity.com/paper/2201.03246/) (2022)
- [er.autopilot 1.0: The Full Autonomous Stack for Oval Racing at High Speeds](https://mlsanity.com/paper/2310.18112/) (2023)
- [PLM-Net: Perception Latency Mitigation Network for Vision-Based Lateral Control of Autonomous Vehicles](https://mlsanity.com/paper/2407.16740/) (2024)
- [Enhancing Robot Learning through Learned Human-Attention Feature Maps](https://mlsanity.com/paper/2308.15327/) (2023)
- [LineMarkNet: Line Landmark Detection for Valet Parking](https://mlsanity.com/paper/2309.10475/) (2023)

## Abstract (arXiv)

Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which it raced. The key components of DUT18D include YOLOv3-based object detection, pose estimation, and time synchronization on its dual stereovision/monovision camera setup. We highlight modifications required to adapt perception CNNs to racing domains, improvements to loss functions used for pose estimation, and methodologies for sub-microsecond camera synchronization among other improvements. We perform a thorough experimental evaluation of the system, demonstrating its accuracy and low-latency in real-world racing scenarios.