Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation
On the map: End-to-end autonomous driving → Generative simulation for driving
Abstract (arXiv)
3D simulation platforms are critical for autonomous driving because they enable end-to-end policy evaluation, thereby reducing development costs and improving safety. In recent years, neural simulation has become predominant, with methods such as NuRec playing a central role; however, these methods remain relatively slow and typically require per-scene tuning. In this work, we present Instant NuRec, a feed-forward neural reconstruction model that turns a short multi-view driving log into a fully simulatable 3D Gaussian Splatting (3DGS) world in a single forward pass. The model accepts multi-view input from a calibrated camera rig and emits a layered output consisting of static and dynamic 3DGS layers, a sky cubemap, and per-camera ISP corrections, while providing native support for non-pinhole camera models via 3DGUT. It reconstructs a 10-20-second multi-camera scene in roughly 1.5 seconds and achieves a PSNR on the Waymo Open Dataset that is 2.01 dB above the strongest evaluated baseline. Instant NuRec is deeply integrated into NuRec and is compatible with AlpaSim for closed-loop simulation.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 69 of its 71 references that are in the map. First posted on arXiv 15 Jul 2026.
Builds on
69 of 71 references.
Closest ancestors · 13
Same direction, newest first
- 2026ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction8 Mar
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models10 Jun
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
- 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
- 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration29 Nov
- 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation17 Oct
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
Other references · 56
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,170
- 2022Scalable Diffusion Models with Transformerscited by 5,973
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,440
- 2021Vision Transformers for Dense Predictioncited by 1,795
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,539
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,486
- 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,231
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 931
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 848
38 more
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 753
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 732
- 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 721
- 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 673
- 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 663
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 658
- 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 557
- 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 349
- 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 345
- 2020Neural Scene Graphs for Dynamic Scenescited by 298
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 277
- 2021Urban Radiance Fieldscited by 263
- 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
- 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
- 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 243
- 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 238
- 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
- 2023EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervisioncited by 217
- 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 166
- 2025Native and Compact Structured Latents for 3D Generationcited by 164
- 2024OmniRe: Omni Urban Scene Reconstructioncited by 160
- 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
- 2023NeuRAD: Neural Rendering for Autonomous Drivingcited by 147
- 2023Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Renderingcited by 143
- 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 141
- 20243D Gaussian Ray Tracing: Fast Tracing of Particle Scenescited by 138
- 20243DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splattingcited by 88
- 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 77
- 2024DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Inputcited by 53
- 2024FreeVS: Generative View Synthesis on Free Driving Trajectorycited by 53
- 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53
- 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
- 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
- 2026V-DPM: 4D Video Reconstruction with Dynamic Point Mapscited by 31
- 2025DGGT: Feedforward 4D Reconstruction of Dynamic Driving Scenes using Unposed Imagescited by 22
- 20264RC: 4D Reconstruction via Conditional Querying Anytime and Anywherecited by 19
- 2026TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokenscited by 12
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation
- 2026DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing
- 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving
- 2025DrivingScene: A Multi-Task Online Feed-Forward 3D Gaussian Splatting Method for Dynamic Driving Scenes
- 2025Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey
- 2025Industrial-Grade Sensor Simulation via Gaussian Splatting: A Modular Framework for Scalable Editing and Full-Stack Validation
- 2025R3D2: Realistic 3D Asset Insertion via Diffusion for Autonomous Driving Simulation
- 2026Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes