mlsanityData to 14 Oct 2026

SCube: Instant Large-Scale Scene Reconstruction using VoxSplats

arXiv 2410.20030 ↗First posted 26 Oct 2024cs.CV cs.AI cs.GRcited by 60

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present SCube, a novel method for reconstructing large-scale 3D scenes (geometry, appearance, and semantics) from a sparse set of posed images. Our method encodes reconstructed scenes using a novel representation VoxSplat, which is a set of 3D Gaussians supported on a high-resolution sparse-voxel scaffold. To reconstruct a VoxSplat from images, we employ a hierarchical voxel latent diffusion model conditioned on the input images followed by a feedforward appearance prediction model. The diffusion model generates high-resolution grids progressively in a coarse-to-fine manner, and the appearance network predicts a set of Gaussians within each voxel. From as few as 3 non-overlapping input images, SCube can generate millions of Gaussians with a 1024^3 voxel grid spanning hundreds of meters in 20 seconds. Past works tackling scene reconstruction from images either rely on per-scene optimization and fail to reconstruct the scene away from input views (thus requiring dense view coverage as input) or leverage geometric priors based on low-resolution models, which produce blurry results. In contrast, SCube leverages high-resolution sparse networks and produces sharp outputs from few views. We show the superiority of SCube compared to prior art using the Waymo self-driving dataset on 3D reconstruction and demonstrate its applications, such as LiDAR simulation and text-to-scene generation.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 68 of its 78 references that are in the map. First posted on arXiv 26 Oct 2024.

Builds on

68 of 78 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
  2. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  3. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  4. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  5. 2023Text-to-3D with Classifier Score Distillation30 Oct
  6. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  8. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 60

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  7. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  9. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  11. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
38 more
  1. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  2. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  3. 2020Convolutional Occupancy Networkscited by 761
  4. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
  5. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  6. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
  7. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  8. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  9. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  10. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  11. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
  12. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  13. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  14. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 405
  15. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  16. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 360
  17. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  18. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  19. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  20. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  21. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  22. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  23. 2024Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussianscited by 230
  24. 2017Submanifold Sparse Convolutional Networkscited by 223
  25. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  26. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  27. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
  28. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  29. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 139
  30. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 130
  31. 2024One-Step Image Translation with Text-to-Image Modelscited by 116
  32. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  33. 2024MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereocited by 87
  34. 2024AbsGS: Recovering Fine Details for 3D Gaussian Splattingcited by 82
  35. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion modelcited by 75
  36. 2022TorchSparse: Efficient Point Cloud Inference Enginecited by 73
  37. 2021Subdivision-Based Mesh Convolution Networkscited by 60
  38. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 28; 2026: 25.

60 papers.

Sort:

By area:

  1. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  2. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  3. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 161
  4. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 84
  5. 2025Bolt3D: Generating 3D Scenes in Secondscited by 59
  6. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 56
  7. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 49
  8. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  9. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  10. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 29
  11. 2024Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstructioncited by 28
  12. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
38 more
  1. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  2. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priorscited by 19
  3. 2024LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understandingcited by 17
  4. 2025GeoPredict: Leveraging Predictive Kinematics and 3D Gaussian Geometry for Precise VLA Manipulationcited by 17
  5. 2025Sat2City: 3D City Generation from A Single Satellite Image with Cascaded Latent Diffusioncited by 16
  6. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusioncited by 15
  7. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 14
  8. 2025WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Drivingcited by 13
  9. 2025InstaInpaint: Instant 3D-Scene Inpainting with Masked Large Reconstruction Modelcited by 10
  10. 2025Flux4D: Flow-based Unsupervised 4D Reconstructioncited by 9
  11. 20254D Driving Scene Generation With Stereo Forcingcited by 8
  12. 2025DiMeR: Disentangled Mesh Reconstruction Modelcited by 6
  13. 2025EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent Diffusioncited by 6
  14. 2026ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Modelscited by 6
  15. 2025Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splattingcited by 5
  16. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  17. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  18. 2025Holistic Large-Scale Scene Reconstruction via Mixed Gaussian Splattingcited by 4
  19. 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Modelcited by 4
  20. 2025OmniView: An All-Seeing Diffusion Model for 3D and 4D View Synthesiscited by 4
  21. 2026GenAssets: Generating in-the-wild 3D Assets in Latent Spacecited by 4
  22. 2025ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusioncited by 3
  23. 2025Voxify3D: Pixel Art Meets Volumetric Renderingcited by 3
  24. 2026WorldFlow3D: Flowing Through 3D Distributions for Unbounded World Generationcited by 3
  25. 2026Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Drivingcited by 3
  26. 2026Sat2City v2: Native 3D City Asset Generation from a Single Satellite Imagecited by 3
  27. 2026Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstructioncited by 2
  28. 2025XYZCylinder: Towards Compatible Feed-Forward 3D Gaussian Splatting for Driving Scenes via Unified Cylinder Lifting Methodcited by 1
  29. 2026ScenDi: 3D-to-2D Scene Diffusion Cascades for Urban Generationcited by 1
  30. 2026Map2World: Segment Map Conditioned Text to 3D World Generationcited by 1
  31. 2026Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenescited by 1
  32. 2026Z-Order Transformer for Feed-Forward Gaussian Splattingcited by 1
  33. 2026Diffusion-guided Generalizable Enhancer for Urban Scene Reconstructioncited by 1
  34. 2026TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenizationcited by 1
  35. 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolutioncited by 1
  36. 2026Sparse auto-regressive modeling for scene generation from multi-view imagescited by 1
  37. 2024ExCellGen: Fast, Controllable, Photorealistic 3D Scene Generation from a Single Real-World Exemplarcited by 0
  38. 2024GSAVS: Gaussian Splatting-based Autonomous Vehicle Simulatorcited by 0

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors
  2. 2026FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representation
  3. 2026VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors
  4. 2026MSVS-VAE: Multi-Scale Anchored VecSet for High-Fidelity 3D Reconstruction
  5. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views
  6. 2025SRSplat: Feed-Forward Super-Resolution Gaussian Splatting from Sparse Multi-View Images
  7. 2024LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors
  8. 2025GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors