# SCaRL- A Synthetic Multi-Modal Dataset for Autonomous Driving

arXiv: https://arxiv.org/abs/2405.17030 · HTML: https://mlsanity.com/paper/2405.17030/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 17 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 4 of 4)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Semantic Pose using Deep Networks Trained on Synthetic RGB-D](https://mlsanity.com/paper/1508.00835/) (2015)

## Built on by (most cited first, 4 of 4)

- [Driving on Registers](https://mlsanity.com/paper/2601.05083/) (2026)
- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [Training Self-Supervised Depth Completion Using Sparse Measurements and a Single Image](https://mlsanity.com/paper/2507.14845/) (2025)
- [SynDORBench: Evaluating LVLM Perceptual Robustness Under Physically Constrained Visibility Conditions](https://mlsanity.com/paper/2609.31823/) (2026)

## Abstract (arXiv)

We present a novel synthetically generated multi-modal dataset, SCaRL, to enable the training and validation of autonomous driving solutions. Multi-modal datasets are essential to attain the robustness and high accuracy required by autonomous systems in applications such as autonomous driving. As deep learning-based solutions are becoming more prevalent for object detection, classification, and tracking tasks, there is great demand for datasets combining camera, lidar, and radar sensors. Existing real/synthetic datasets for autonomous driving lack synchronized data collection from a complete sensor suite. SCaRL provides synchronized Synthetic data from RGB, semantic/instance, and depth Cameras; Range-Doppler-Azimuth/Elevation maps and raw data from Radar; and 3D point clouds/2D maps of semantic, depth and Doppler data from coherent Lidar. SCaRL is a large dataset based on the CARLA Simulator, which provides data for diverse, dynamic scenarios and traffic conditions. SCaRL is the first dataset to include synthetic synchronized data from coherent Lidar and MIMO radar sensors. The dataset can be accessed here: https://fhr-ihs-sva.pages.fraunhofer.de/asp/scarl/