# DrivIng: A Large-Scale Multimodal Driving Dataset with Full Digital Twin Integration

arXiv: https://arxiv.org/abs/2601.15260 · HTML: https://mlsanity.com/paper/2601.15260/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving](https://mlsanity.com/paper/2305.02008/) (2023)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [CitySim: A Drone-Based Vehicle Trajectory Dataset for Safety Oriented Research and Digital Twins](https://mlsanity.com/paper/2208.11036/) (2022)
- [UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perception](https://mlsanity.com/paper/2510.23478/) (2025)
- [TWICE Dataset: Digital Twin of Test Scenarios in a Controlled Environment](https://mlsanity.com/paper/2310.03895/) (2023)
- [Unlocking Past Information: Temporal Embeddings in Cooperative Bird's Eye View Prediction](https://mlsanity.com/paper/2401.14325/) (2024)
- [Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach](https://mlsanity.com/paper/2507.17070/) (2025)

## Built on by (most cited first, 3 of 3)

- [Real-Time Evaluation of Autonomous Systems under Adversarial Attacks](https://mlsanity.com/paper/2605.03491/) (2026)
- [RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/2606.29867/) (2026)
- [Learning-Based Behavior Planning for Automated Driving: Real-World Integration and Deployment](https://mlsanity.com/paper/2608.12198/) (2026)

## Abstract (arXiv)

Perception is a cornerstone of autonomous driving, enabling vehicles to understand their surroundings and make safe, reliable decisions. Developing robust perception algorithms requires large-scale, high-quality datasets that cover diverse driving conditions and support thorough evaluation. Existing datasets often lack a high-fidelity digital twin, limiting systematic testing, edge-case simulation, sensor modification, and sim-to-real evaluations. To address this gap, we present DrivIng, a large-scale multimodal dataset with a complete geo-referenced digital twin of a ~18 km route spanning urban, suburban, and highway segments. Our dataset provides continuous recordings from six RGB cameras, one LiDAR, and high-precision ADMA-based localization, captured across day, dusk, and night. All sequences are annotated at 10 Hz with 3D bounding boxes and track IDs across 12 classes, yielding ~1.2 million annotated instances. Alongside the benefits of a digital twin, DrivIng enables a 1-to-1 transfer of real traffic into simulation, preserving agent interactions while enabling realistic and flexible scenario testing. To support reproducible research and robust validation, we benchmark DrivIng with state-of-the-art perception models and publicly release the dataset, digital twin, HD map, and codebase.