mlsanityData to 7 Oct 2026

GS-Net: Heterogeneous Vehicle Data Reuse via Generalizable Plug-and-Play 3DGS Module

arXiv 2409.11307 ↗First posted 17 Sep 2024cs.CVcited by 0

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

End-to-end autonomous driving is increasingly data-driven, yet data reuse across vehicles remains limited. Each new vehicle often requires additional data collection and retraining because camera translation, orientation, and field of view differ across sensor layouts. Cross-sensor view synthesis offers a promising route for cross-platform data reuse by synthesizing images under novel sensor configurations from existing sensor data. To realize this goal, we propose GS-Net, a lightweight plug-and-play module that aggregates local geometric context from sparse Structure-from-Motion (SfM) point clouds and expands each point into multiple dense Gaussian primitives in a single forward pass, learning a cross-scene generalizable initialization for standard 3DGS that improves rendering quality for both interpolated views along the original sensor trajectories and extrapolated views at new sensor positions. In such settings, target camera viewpoints often lie beyond the convex hull of training cameras, whereas existing methods and benchmarks predominantly evaluate in-hull interpolation, leaving the extrapolation regime underexplored. To enable quantitative evaluation under this setting, we introduce CARLA-NVS, the first benchmark explicitly designed for cross-sensor view synthesis. Unlike existing autonomous driving datasets that typically employ no more than eight cameras in fixed configurations, CARLA-NVS features 12 cameras uniformly distributed at 30-degree azimuth intervals, enabling controlled evaluation of both interpolated and extrapolated viewpoints. Experiments on CARLA-NVS show that GS-Net improves rendering quality by 2.08 dB PSNR on interpolated views and 1.86 dB on extrapolated views over standard 3DGS, while achieving a 50x faster initialization compared to MVS-based densification. These results offer a practical step toward scalable cross-vehicle data reuse in autonomous driving.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 12 of its 31 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

12 of 31 references.

References · 12

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2024Depth Anything V2cited by 1,385
  7. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  8. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  9. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  10. 2024GaussianPro: 3D Gaussian Splatting with Progressive Propagationcited by 199
  11. 2024InstantSplat: Sparse-view Gaussian Splatting in Secondscited by 67
  12. 2024GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-timecited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving
  2. 2025I2V-GS: Infrastructure-to-Vehicle View Transformation with Gaussian Splatting for Autonomous Driving Data Generation
  3. 2026Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis
  4. 2026ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction
  5. 2026L2D2-GS: Learning to Densify for Feedforward Dynamic Gaussian Scene Reconstruction
  6. 2026Robust 4D Driving Scene Reconstruction from Imperfect Visual Priors
  7. 2025MuDG: Taming Multi-modal Diffusion with Gaussian Splatting for Urban Scene Reconstruction
  8. 2026Towards Physically Consistent 4D Scene Reconstruction for Closed-loop Autonomous Driving Simulation