# Challenges in Designing Datasets and Validation for Autonomous Driving

arXiv: https://arxiv.org/abs/1901.09270 · HTML: https://mlsanity.com/paper/1901.09270/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2019.

## Built on by (most cited first, 23 of 23)

- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2103.17001/) (2021)
- [Weather and Light Level Classification for Autonomous Driving: Dataset, Baseline and Active Learning](https://mlsanity.com/paper/2104.14042/) (2021)
- [Navya3DSeg -- Navya 3D Semantic Segmentation Dataset & split generation for autonomous vehicles](https://mlsanity.com/paper/2302.08292/) (2023)
- [An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications](https://mlsanity.com/paper/2301.13476/) (2023)
- [Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022 OmniCV Workshop Challenge](https://mlsanity.com/paper/2206.12912/) (2022)
- [FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception](https://mlsanity.com/paper/2604.10391/) (2026)
- [Ensemble-based Semi-supervised Learning to Improve Noisy Soiling Annotations in Autonomous Driving](https://mlsanity.com/paper/2105.07930/) (2021)
- [Surround-View Cameras based Holistic Visual Perception for Automated Driving](https://mlsanity.com/paper/2206.05542/) (2022)
- [LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Driving](https://mlsanity.com/paper/2307.08850/) (2023)
- [Multi-camera Bird's Eye View Perception for Autonomous Driving](https://mlsanity.com/paper/2309.09080/) (2023)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [FisheyeDetNet: 360deg Surround view Fisheye Camera based Object Detection System for Autonomous Driving](https://mlsanity.com/paper/2404.13443/) (2024)
- [SimBEV: A Synthetic Multi-Task Multi-Sensor Driving Data Generation Tool and Dataset](https://mlsanity.com/paper/2502.01894/) (2025)
- [Evaluating the effect of data augmentation and BALD heuristics on distillation of Semantic-KITTI dataset](https://mlsanity.com/paper/2302.10679/) (2023)
- [WoodScape Motion Segmentation for Autonomous Driving -- CVPR 2023 OmniCV Workshop Challenge](https://mlsanity.com/paper/2401.00910/) (2023)
- [Neural Rendering based Urban Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2402.06826/) (2024)
- [Dataset Safety in Autonomous Driving: Requirements, Risks, and Assurance](https://mlsanity.com/paper/2511.08439/) (2025)
- [Neuromorphic LiDAR-based Bird's Eye View Object Detection using Energy-efficient Spiking Neural Networks](https://mlsanity.com/paper/2605.25293/) (2026)
- [SimBEV2X: A Large-Scale Dataset and Data Generation Tool for Multi-Task Vehicle-to-Everything Cooperative Perception](https://mlsanity.com/paper/2607.23910/) (2026)
- [Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving](https://mlsanity.com/paper/2608.20874/) (2026)
- [RedLight-VLA: Models for traffic-rule grounding and behavioral emphasis in driving policies](https://mlsanity.com/paper/2608.28656/) (2026)
- [MotionSync: Non-Causal Refinement of Causal Tracker for Label-Efficient 3D Perception](https://mlsanity.com/paper/2608.29567/) (2026)

## Abstract (arXiv)

Autonomous driving is getting a lot of attention in the last decade and will be the hot topic at least until the first successful certification of a car with Level 5 autonomy. There are many public datasets in the academic community. However, they are far away from what a robust industrial production system needs. There is a large gap between academic and industrial setting and a substantial way from a research prototype, built on public datasets, to a deployable solution which is a challenging task. In this paper, we focus on bad practices that often happen in the autonomous driving from an industrial deployment perspective. Data design deserves at least the same amount of attention as the model design. There is very little attention paid to these issues in the scientific community, and we hope this paper encourages better formalization of dataset design. More specifically, we focus on the datasets design and validation scheme for autonomous driving, where we would like to highlight the common problems, wrong assumptions, and steps towards avoiding them, as well as some open problems.