# XLD: A Cross-Lane Dataset for Benchmarking Novel Driving View Synthesis arXiv: https://arxiv.org/abs/2406.18360 ยท HTML: https://mlsanity.com/paper/2406.18360/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. - Builds on 48 of its 67 references that are in the map. - First posted on arXiv 26 Jun 2024. ## Placed in - End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/) ## Builds on (most cited first, 30 of 48) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016) - [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022) - [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021) - [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019) - [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017) - [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021) - [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021) - [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018) - [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022) - [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019) - [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022) - [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023) - [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023) - [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023) - [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023) - [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021) - [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024) - [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017) - [DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes](https://mlsanity.com/paper/2312.07920/) (2023) - [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020) - [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019) - [Neural Scene Graphs for Dynamic Scenes](https://mlsanity.com/paper/2011.10379/) (2020) - [UniSim: A Neural Closed-Loop Sensor Simulator](https://mlsanity.com/paper/2308.01898/) (2023) - [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019) - [GenAD: Generative End-to-End Autonomous Driving](https://mlsanity.com/paper/2402.11502/) (2024) - [EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision](https://mlsanity.com/paper/2311.02077/) (2023) - [A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets](https://mlsanity.com/paper/2406.12080/) (2024) - [GaussianPro: 3D Gaussian Splatting with Progressive Propagation](https://mlsanity.com/paper/2402.14650/) (2024) ## Built on by (most cited first, 9 of 9) - [DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation](https://mlsanity.com/paper/2410.13571/) (2024) - [ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration](https://mlsanity.com/paper/2411.19548/) (2024) - [CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2503.23044/) (2025) - [Para-Lane: Multi-Lane Dataset Registering Parallel Scans for Benchmarking Novel View Synthesis](https://mlsanity.com/paper/2502.15635/) (2025) - [Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2503.14537/) (2025) - [SEED4D: A Synthetic Ego--Exo Dynamic 4D Data Generator, Driving Dataset and Benchmark](https://mlsanity.com/paper/2412.00730/) (2024) - [Prioritizing Perception-Guided Self-Supervision: A New Paradigm for Causal Modeling in End-to-End Autonomous Driving](https://mlsanity.com/paper/2511.08214/) (2025) - [HybridWorldSim: A Scalable and Controllable High-fidelity Simulator for Autonomous Driving](https://mlsanity.com/paper/2511.22187/) (2025) - [MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis](https://mlsanity.com/paper/2608.12442/) (2026) ## Abstract (arXiv) Comprehensive testing of autonomous systems through simulation is essential to ensure the safety of autonomous driving vehicles. This requires the generation of safety-critical scenarios that extend beyond the limitations of real-world data collection, as many of these scenarios are rare or rarely encountered on public roads. However, evaluating most existing novel view synthesis (NVS) methods relies on sporadic sampling of image frames from the training data, comparing the rendered images with ground-truth images. Unfortunately, this evaluation protocol falls short of meeting the actual requirements in closed-loop simulations. Specifically, the true application demands the capability to render novel views that extend beyond the original trajectory (such as cross-lane views), which are challenging to capture in the real world. To address this, this paper presents a synthetic dataset for novel driving view synthesis evaluation, which is specifically designed for autonomous driving simulations. This unique dataset includes testing images captured by deviating from the training trajectory by 1-4 meters. It comprises six sequences that cover various times and weather conditions. Each sequence contains 450 training images, 120 testing images, and their corresponding camera poses and intrinsic parameters. Leveraging this novel dataset, we establish the first realistic benchmark for evaluating existing NVS approaches under front-only and multicamera settings. The experimental findings underscore the significant gap in current approaches, revealing their inadequate ability to fulfill the demanding prerequisites of cross-lane or closed-loop simulation.