# UNet-Based Keypoint Regression for 3D Cone Localization in Autonomous Racing

arXiv: https://arxiv.org/abs/2602.21904 · HTML: https://mlsanity.com/paper/2602.21904/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 25 Feb 2026.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Deep Learning vs. Traditional Computer Vision](https://mlsanity.com/paper/1910.13796/) (2019)
- [Fast Human Pose Estimation](https://mlsanity.com/paper/1811.05419/) (2018)
- [YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss](https://mlsanity.com/paper/2204.06806/) (2022)
- [Worsening Perception: Real-time Degradation of Autonomous Vehicle Perception Performance for Simulation of Adverse Weather Conditions](https://mlsanity.com/paper/2103.02760/) (2021)
- [CenterNet++ for Object Detection](https://mlsanity.com/paper/2204.08394/) (2022)
- [Real-time 3D Traffic Cone Detection for Autonomous Driving](https://mlsanity.com/paper/1902.02394/) (2019)
- [FSOCO: The Formula Student Objects in Context Dataset](https://mlsanity.com/paper/2012.07139/) (2020)
- [Joint Coordinate Regression and Association For Multi-Person Pose Estimation, A Pure Neural Network Approach](https://mlsanity.com/paper/2307.01004/) (2023)

## Abstract (arXiv)

Accurate cone localization in 3D space is essential in autonomous racing for precise navigation around the track. Approaches that rely on traditional computer vision algorithms are sensitive to environmental variations, and neural networks are often trained on limited data and are infeasible to run in real time. We present a UNet-based neural network for keypoint detection on cones, leveraging the largest custom-labeled dataset we have assembled. Our approach enables accurate cone position estimation and the potential for color prediction. Our model achieves substantial improvements in keypoint accuracy over conventional methods. Furthermore, we leverage our predicted keypoints in the perception pipeline and evaluate the end-to-end autonomous system. Our results show high-quality performance across all metrics, highlighting the effectiveness of this approach and its potential for adoption in competitive autonomous racing systems.