Shell-Supervised Gaussian Splatting for Urban Real-to-Sim Reconstruction
Not on the map. Why?
Abstract (arXiv)
Real-to-sim reconstruction for embodied AI requires geometry that is useful for collision reasoning, navigation, and agent-environment interaction, not only photorealistic novel-view synthesis. However, close-range urban facades are difficult for video-to-3D reconstruction: glass, reflections, repeated windows, and weak texture can produce visually plausible renderings with unstable surface geometry. We introduce shell-supervised Gaussian Splatting, a reconstruction-stage framework that uses an external facade structural shell as lightweight geometric supervision for video-driven Gaussian reconstruction. The method aligns an exterior shell to the video reconstruction frame, renders per-view depth, camera-space normal, and valid-mask maps, and applies these cues through mask-gated losses during Gaussian optimization. This design preserves RGB-driven appearance while regularizing only visible shell-supported facade regions. Experiments on anonymized close-range urban facade scenes show improved facade orientation and visible-surface point-cloud consistency over photo-only, monocular-cue, and surface-oriented Gaussian baselines, while maintaining comparable held-out rendering quality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 46 references that are in the map. First posted on arXiv 29 Jun 2026.
Builds on
38 of 46 references.
References · 38
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
- 2021Volume Rendering of Neural Implicit Surfacescited by 996
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
26 more
- 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 724
- 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
- 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
- 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
- 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
- 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 467
- 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 398
- 2021Urban Radiance Fieldscited by 263
- 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
- 2024Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenescited by 232
- 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 221
- 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 155
- 2024DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshingcited by 138
- 2022NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priorscited by 125
- 2022Neural 3D Scene Reconstruction with the Manhattan-world Assumptioncited by 123
- 2023StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Viewscited by 120
- 2023S-NeRF: Neural Radiance Fields for Street Viewscited by 116
- 2024Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splattingcited by 113
- 2025Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigationcited by 30
- 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
- 2024Geometry Field Splatting with Gaussian Surfelscited by 13
- 2024Planar Gaussian Splattingcited by 6
- 2025GS4Buildings: Prior-Guided Gaussian Splatting for 3D Building Reconstructioncited by 5
- 2024BIMCaP: BIM-based AI-supported LiDAR-Camera Pose Refinementcited by 1
Built on by
Not cited yet.