# ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings

arXiv: https://arxiv.org/abs/2410.20686 · HTML: https://mlsanity.com/paper/2410.20686/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 29 of 29)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2311.13384/) (2023)
- [Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images](https://mlsanity.com/paper/2311.13398/) (2023)
- [DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting](https://mlsanity.com/paper/2404.06903/) (2024)
- [OpenVSLAM: A Versatile Visual SLAM Framework](https://mlsanity.com/paper/1910.01122/) (2019)
- [MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images](https://mlsanity.com/paper/2008.06534/) (2020)
- [On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy](https://mlsanity.com/paper/2402.00752/) (2024)
- [360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming](https://mlsanity.com/paper/2402.00763/) (2024)
- [360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries](https://mlsanity.com/paper/2311.17389/) (2023)
- [Balanced Spherical Grid for Egocentric View Synthesis](https://mlsanity.com/paper/2303.12408/) (2023)
- [Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network](https://mlsanity.com/paper/1907.00939/) (2019)
- [360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidance](https://mlsanity.com/paper/2209.14265/) (2022)
- [Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama](https://mlsanity.com/paper/2106.10859/) (2021)
- [OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems](https://mlsanity.com/paper/2003.08056/) (2020)
- [PICCOLO: Point Cloud-Centric Omnidirectional Localization](https://mlsanity.com/paper/2108.06545/) (2021)
- [Indoor Panorama Planar 3D Reconstruction via Divide and Conquer](https://mlsanity.com/paper/2106.14166/) (2021)
- [OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas](https://mlsanity.com/paper/2104.09403/) (2021)
- [Casual 6-DoF: free-viewpoint panorama using a handheld 360 camera](https://mlsanity.com/paper/2203.16756/) (2022)
- [OmniVoxel: A Fast and Precise Reconstruction Method of Omnidirectional Neural Radiance Field](https://mlsanity.com/paper/2208.06335/) (2022)

## Built on by (most cited first, 20 of 20)

- [Matrix-3D: Omnidirectional Explorable 3D World Generation](https://mlsanity.com/paper/2508.08086/) (2025)
- [OmniSplat: Taming Feed-Forward 3D Gaussian Splatting for Omnidirectional Images with Editable Capabilities](https://mlsanity.com/paper/2412.16604/) (2024)
- [Seam360GS: Seamless 360deg Gaussian Splatting from Real-World Omnidirectional Images](https://mlsanity.com/paper/2508.20080/) (2025)
- [ErpGS: Equirectangular Image Rendering enhanced with 3D Gaussian Regularization](https://mlsanity.com/paper/2505.19883/) (2025)
- [PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding](https://mlsanity.com/paper/2512.20907/) (2025)
- [You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting](https://mlsanity.com/paper/2504.09062/) (2025)
- [OB3D: A New Dataset for Benchmarking Omnidirectional 3D Reconstruction Using Blender](https://mlsanity.com/paper/2505.20126/) (2025)
- [PanoImager: Geometry-Guided Novel View Synthesis and Reconstruction from Sparse Panoramic Views](https://mlsanity.com/paper/2606.27071/) (2026)
- [Spherical-GOF: Geometry-Aware Panoramic Gaussian Opacity Fields for 3D Scene Reconstruction](https://mlsanity.com/paper/2603.08503/) (2026)
- [FullCircle: Effortless 3D Reconstruction from Casual 360^circ Captures](https://mlsanity.com/paper/2603.22572/) (2026)
- [Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors](https://mlsanity.com/paper/2603.23324/) (2026)
- [Underwater360: Reconstructing Underwater Scenes from Panoramic Images with Omnidirectional Gaussian Splatting](https://mlsanity.com/paper/2605.26447/) (2026)
- [UniTriSplat: A Unified 3D Gaussian Splatting Framework with Uniform Spherical Rasterization for Universal Cameras](https://mlsanity.com/paper/2606.29794/) (2026)
- [FastPano3D: Feed-Forward Indoor Panoramic 3D Reconstruction from a Single Image](https://mlsanity.com/paper/2606.30352/) (2026)
- [Unified Panoramic-Gaussian Representation for Monocular 4D Scene Synthesis](https://mlsanity.com/paper/2607.01663/) (2026)
- [Geometry and Gradient-based Partitioning for Panoramic Outdoor Reconstruction](https://mlsanity.com/paper/2607.08769/) (2026)
- [PanoGS-SLAM: Panoramic 3D Gaussian Splatting SLAM](https://mlsanity.com/paper/2609.17387/) (2026)
- [Cube-Splat: High-Fidelity 360deg Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization](https://mlsanity.com/paper/2609.21347/) (2026)
- [PanOVOcc: Panoramic Embodied Open-Vocabulary Occupancy Mapping with Long-term Spatial Voxel Memory](https://mlsanity.com/paper/2609.31716/) (2026)
- [Feedforward Novel View Synthesis for Heterogeneous Cameras](https://mlsanity.com/paper/2610.03522/) (2026)

## Abstract (arXiv)

Omnidirectional (or 360-degree) images are increasingly being used for 3D applications since they allow the rendering of an entire scene with a single image. Existing works based on neural radiance fields demonstrate successful 3D reconstruction quality on egocentric videos, yet they suffer from long training and rendering times. Recently, 3D Gaussian splatting has gained attention for its fast optimization and real-time rendering. However, directly using a perspective rasterizer to omnidirectional images results in severe distortion due to the different optical properties between two image domains. In this work, we present ODGS, a novel rasterization pipeline for omnidirectional images, with geometric interpretation. For each Gaussian, we define a tangent plane that touches the unit sphere and is perpendicular to the ray headed toward the Gaussian center. We then leverage a perspective camera rasterizer to project the Gaussian onto the corresponding tangent plane. The projected Gaussians are transformed and combined into the omnidirectional image, finalizing the omnidirectional rasterization process. This interpretation reveals the implicit assumptions within the proposed pipeline, which we verify through mathematical proofs. The entire rasterization process is parallelized using CUDA, achieving optimization and rendering speeds 100 times faster than NeRF-based methods. Our comprehensive experiments highlight the superiority of ODGS by delivering the best reconstruction and perceptual quality across various datasets. Additionally, results on roaming datasets demonstrate that ODGS restores fine details effectively, even when reconstructing large 3D scenes. The source code is available on our project page (https://github.com/esw0116/ODGS).