mlsanityData to 14 Oct 2026

3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting

arXiv 2412.12507 ↗First posted 17 Dec 2024cs.GR cs.CVcited by 88

Not on the map. Why?

Abstract (arXiv)

3D Gaussian Splatting (3DGS) enables efficient reconstruction and high-fidelity real-time rendering of complex scenes on consumer hardware. However, due to its rasterization-based formulation, 3DGS is constrained to ideal pinhole cameras and lacks support for secondary lighting effects. Recent methods address these limitations by tracing the particles instead, but, this comes at the cost of significantly slower rendering. In this work, we propose 3D Gaussian Unscented Transform (3DGUT), replacing the EWA splatting formulation with the Unscented Transform that approximates the particles through sigma points, which can be projected exactly under any nonlinear projection function. This modification enables trivial support of distorted cameras with time dependent effects such as rolling shutter, while retaining the efficiency of rasterization. Additionally, we align our rendering formulation with that of tracing-based methods, enabling secondary ray tracing required to represent phenomena such as reflections and refraction within the same 3D representation. The source code is available at: https://github.com/nv-tlabs/3dgrut.

Key facts. Cited by 88 papers in the map (data through 14 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

44 of 58 references.

References · 44

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  5. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  6. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  7. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  9. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  10. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  11. 2020Neural Sparse Voxel Fieldscited by 959
  12. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
32 more
  1. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
  2. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  3. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
  4. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
  5. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 511
  6. 2023Compact 3D Gaussian Representation for Radiance Fieldcited by 372
  7. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 311
  8. 2020Free View Synthesiscited by 231
  9. 2023BakedSDF: Meshing Neural SDFs for Real-Time View Synthesiscited by 221
  10. 2023MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenescited by 218
  11. 20243D Gaussian Splatting as Markov Chain Monte Carlocited by 217
  12. 2024A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasetscited by 216
  13. 2024VastGaussian: Vast 3D Gaussians for Large Scene Reconstructioncited by 209
  14. 2024Taming 3DGS: High-Quality Radiance Fields with Limited Resourcescited by 204
  15. 2024CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussianscited by 192
  16. 2021Point-Based Neural Rendering with Per-View Optimizationcited by 190
  17. 2024OmniRe: Omni Urban Scene Reconstructioncited by 161
  18. 20243D Gaussian Ray Tracing: Fast Tracing of Particle Scenescited by 139
  19. 2024StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Renderingcited by 111
  20. 2024AbsGS: Recovering Fine Details for 3D Gaussian Splattingcited by 82
  21. 2023Adaptive Shells for Efficient Neural Radiance Field Renderingcited by 69
  22. 2023SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Explorationcited by 62
  23. 2024EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesiscited by 48
  24. 2024On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategycited by 39
  25. 2024Don't Splat your Gaussians: Volumetric Ray-Traced Primitives for Modeling and Rendering Scattering and Emissive Mediacited by 37
  26. 2024Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparencycited by 35
  27. 2024Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motioncited by 29
  28. 2023Learning Neural Duplex Radiance Fields for Real-Time View Synthesiscited by 28
  29. 2023HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfacescited by 28
  30. 2024Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Camerascited by 24
  31. 2024Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Renderingcited by 19
  32. 2023Volumetric Rendering with Baked Quadrature Fieldscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 41; 2026: 47.

88 papers.

Sort:

By area:

  1. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 146
  2. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 84
  3. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 62
  4. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  5. 2025MILo: Mesh-In-the-Loop Gaussian Splatting for Detailed and Efficient Surface Reconstructioncited by 37
  6. 2025LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Renderingcited by 25
  7. 2026RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policiescited by 24
  8. 2025GeoSVR: Taming Sparse Voxels for Geometrically Accurate Surface Reconstructioncited by 17
  9. 2025Towards Physically Executable 3D Gaussian for Embodied Navigationcited by 17
  10. 2026Faster-GS: Analyzing and Improving Gaussian Splatting Optimizationcited by 17
  11. 2026TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokenscited by 15
  12. 2025TSGS: Improving Gaussian Splatting for Transparent Surface Reconstruction via Normal and De-lighting Priorscited by 14
38 more
  1. 2025AAA-Gaussians: Anti-Aliased and Artifact-Free 3D Gaussian Renderingcited by 12
  2. 2026DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancercited by 10
  3. 2025TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splattingcited by 9
  4. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 9
  5. 20253DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Camerascited by 8
  6. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  7. 2025FIORD: A Fisheye Indoor-Outdoor Dataset with LIDAR Ground Truth for 3D Scene Reconstruction and Benchmarkingcited by 7
  8. 2026ParkGaussian: Surround-view 3D Gaussian Splatting for Autonomous Parkingcited by 7
  9. 2026PPISP: Physically-Plausible Compensation and Control of Photometric Variations in Radiance Field Reconstructioncited by 7
  10. 2025RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policiescited by 6
  11. 2025GSCache: Real-Time Radiance Caching for Volume Path Tracing using 3D Gaussian Splattingcited by 5
  12. 2026Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Drivingcited by 5
  13. 2025Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstructioncited by 4
  14. 2025Universal Beta Splattingcited by 4
  15. 2025SimULi: Real-Time LiDAR and Camera Simulation with Unscented Transformscited by 4
  16. 2026Neural Harmonic Textures for High-Quality Primitive Based Neural Reconstructioncited by 4
  17. 2025SaLF: Sparse Local Fields for Multi-Sensor Rendering in Real-Timecited by 3
  18. 2025Wanderland: Geometrically Grounded Simulation for Open-World Embodied AIcited by 3
  19. 2026Confidence-Based Mesh Extraction from 3D Gaussianscited by 3
  20. 2026Power Foam: Unifying Real-Time Differentiable Ray Tracing and Rasterizationcited by 3
  21. 2026RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modelingcited by 3
  22. 2026Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulationcited by 3
  23. 2026ECoNGS: Efficient Compressive Neural Gaussian Splats for Volume Visualizationcited by 3
  24. 2026Axolotl3D: a Unified Framework for Faithful 3D Shape Completioncited by 3
  25. 2025RealMirror: A Comprehensive, Open-Source Vision-Language-Action Platform for Embodied AIcited by 2
  26. 2025UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Renderingcited by 2
  27. 2025Material-informed Gaussian Splatting for 3D World Reconstruction in a Digital Twincited by 2
  28. 2025Moment-Based 3D Gaussian Splatting: Resolving Volumetric Occlusion with Order-Independent Transmittancecited by 2
  29. 2026Benchmarking PhD-Level Coding in 3D Geometric Computer Visioncited by 2
  30. 2026Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulationcited by 2
  31. 2026HiGS: A Hierarchical Rendering Architecture for Real-Time 3D Gaussian Splattingcited by 2
  32. 2026RPC-GS: Gaussian Splatting with native RPC Rendering for Satellite Imagerycited by 2
  33. 2025RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objectscited by 1
  34. 2025AgriChrono: A Multi-modal Dataset Capturing Crop Growth and Lighting Variability with a Field Robotcited by 1
  35. 2025A Bag of Tricks for Efficient Implicit Neural Point Cloudscited by 1
  36. 2025From Volume Rendering to 3D Gaussian Splatting: Theory and Applicationscited by 1
  37. 2025The Impact and Outlook of 3D Gaussian Splattingcited by 1
  38. 2025FisheyeGaussianLift: BEV Feature Lifting for Surround-View Fisheye Camera Perceptioncited by 1

50 most cited of 88 All 88 (JSON)