# R3ST: A Synthetic 3D Dataset With Realistic Trajectories

arXiv: https://arxiv.org/abs/2512.16784 · HTML: https://mlsanity.com/paper/2512.16784/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Builds on (most cited first, 18 of 18)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning](https://mlsanity.com/paper/2003.02437/) (2020)
- [SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation](https://mlsanity.com/paper/2206.08367/) (2022)
- [Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention](https://mlsanity.com/paper/2210.09071/) (2022)
- [IDDA: a large-scale multi-domain dataset for autonomous driving](https://mlsanity.com/paper/2004.08298/) (2020)
- [SIND: A Drone Dataset at Signalized Intersection in China](https://mlsanity.com/paper/2209.02297/) (2022)
- [Simultaneous Detection and Tracking with Motion Modelling for Multiple Object Tracking](https://mlsanity.com/paper/2008.08826/) (2020)
- [Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator](https://mlsanity.com/paper/2502.19204/) (2025)
- [WARM-3D: A Weakly-Supervised Sim2Real Domain Adaptation Framework for Roadside Monocular 3D Object Detection](https://mlsanity.com/paper/2407.20818/) (2024)
- [A POV-based Highway Vehicle Trajectory Dataset and Prediction Architecture](https://mlsanity.com/paper/2303.06202/) (2023)
- [SynTraC: A Synthetic Dataset for Traffic Signal Control from Traffic Monitoring Cameras](https://mlsanity.com/paper/2408.09588/) (2024)

## Abstract (arXiv)

Datasets are essential to train and evaluate computer vision models used for traffic analysis and to enhance road safety. Existing real datasets fit real-world scenarios, capturing authentic road object behaviors, however, they typically lack precise ground-truth annotations. In contrast, synthetic datasets play a crucial role, allowing for the annotation of a large number of frames without additional costs or extra time. However, a general drawback of synthetic datasets is the lack of realistic vehicle motion, since trajectories are generated using AI models or rule-based systems. In this work, we introduce R3ST (Realistic 3D Synthetic Trajectories), a synthetic dataset that overcomes this limitation by generating a synthetic 3D environment and integrating real-world trajectories derived from SinD, a bird's-eye-view dataset recorded from drone footage. The proposed dataset closes the gap between synthetic data and realistic trajectories, advancing the research in trajectory forecasting of road vehicles, offering both accurate multimodal ground-truth annotations and authentic human-driven vehicle trajectories.