# Beyond the Mud: Datasets and Benchmarks for Computer Vision in Off-Road Racing

arXiv: https://arxiv.org/abs/2402.08025 · HTML: https://mlsanity.com/paper/2402.08025/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 83 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Person Re-identification: Past, Present and Future](https://mlsanity.com/paper/1610.02984/) (2016)
- [Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes](https://mlsanity.com/paper/1908.08207/) (2019)
- [Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition](https://mlsanity.com/paper/1710.10400/) (2017)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [Detecting Text in Natural Image with Connectionist Text Proposal Network](https://mlsanity.com/paper/1609.03605/) (2016)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting](https://mlsanity.com/paper/2211.10772/) (2022)
- [Unsupervised Person Re-identification: Clustering and Fine-tuning](https://mlsanity.com/paper/1705.10444/) (2017)
- [A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector](https://mlsanity.com/paper/1802.09567/) (2018)
- [Open Images V5 Text Annotation and Yet Another Mask Text Spotter](https://mlsanity.com/paper/2106.12326/) (2021)
- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [An Efficient and Layout-Independent Automatic License Plate Recognition System Based on the YOLO detector](https://mlsanity.com/paper/1909.01754/) (2019)
- [CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model](https://mlsanity.com/paper/2305.14014/) (2023)
- [Person Re-Identification by Discriminative Selection in Video Ranking](https://mlsanity.com/paper/1601.06260/) (2016)
- [Multi-task learning for jersey number recognition in Ice Hockey](https://mlsanity.com/paper/2108.07848/) (2021)
- [Learning Resolution-Invariant Deep Representations for Person Re-Identification](https://mlsanity.com/paper/1907.10843/) (2019)
- [Transferring a Semantic Representation for Person Re-Identification and Search](https://mlsanity.com/paper/1706.03725/) (2017)
- [Knock, knock. Who's there? -- Identifying football player jersey numbers with synthetic data](https://mlsanity.com/paper/2203.00734/) (2022)

## Abstract (arXiv)

Despite significant progress in optical character recognition (OCR) and computer vision systems, robustly recognizing text and identifying people in images taken in unconstrained in-the-wild environments remain an ongoing challenge. However, such obstacles must be overcome in practical applications of vision systems, such as identifying racers in photos taken during off-road racing events. To this end, we introduce two new challenging real-world datasets - the off-road motorcycle Racer Number Dataset (RND) and the Muddy Racer re-iDentification Dataset (MUDD) - to highlight the shortcomings of current methods and drive advances in OCR and person re-identification (ReID) under extreme conditions. These two datasets feature over 6,300 images taken during off-road competitions which exhibit a variety of factors that undermine even modern vision systems, namely mud, complex poses, and motion blur. We establish benchmark performance on both datasets using state-of-the-art models. Off-the-shelf models transfer poorly, reaching only 15% end-to-end (E2E) F1 score on text spotting, and 33% rank-1 accuracy on ReID. Fine-tuning yields major improvements, bringing model performance to 53% F1 score for E2E text spotting and 79% rank-1 accuracy on ReID, but still falls short of good performance. Our analysis exposes open problems in real-world OCR and ReID that necessitate domain-targeted techniques. With these datasets and analysis of model limitations, we aim to foster innovations in handling real-world conditions like mud and complex poses to drive progress in robust computer vision. All data was sourced from PerformancePhoto.co, a website used by professional motorsports photographers, racers, and fans. The top-performing text spotting and ReID models are deployed on this platform to power real-time race photo search.