# Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras

arXiv: https://arxiv.org/abs/2409.04751 · HTML: https://mlsanity.com/paper/2409.04751/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 7 Sep 2024.

## Builds on (most cited first, 28 of 28)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization](https://mlsanity.com/paper/2403.06912/) (2024)
- [Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians](https://mlsanity.com/paper/2403.17898/) (2024)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering](https://mlsanity.com/paper/2311.17089/) (2023)
- [SparseGS: Sparse View Synthesis using 3D Gaussian Splatting](https://mlsanity.com/paper/2312.00206/) (2023)
- [FlashGS: Efficient 3D Gaussian Splatting for Large-scale and High-resolution Rendering](https://mlsanity.com/paper/2408.07967/) (2024)
- [Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration](https://mlsanity.com/paper/2403.11056/) (2024)
- [PERF: Panoramic Neural Radiance Field from a Single Panorama](https://mlsanity.com/paper/2310.16831/) (2023)
- [On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy](https://mlsanity.com/paper/2402.00752/) (2024)
- [360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming](https://mlsanity.com/paper/2402.00763/) (2024)
- [360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidance](https://mlsanity.com/paper/2209.14265/) (2022)
- [Neural Lens Modeling](https://mlsanity.com/paper/2304.04848/) (2023)
- [Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges](https://mlsanity.com/paper/2402.12041/) (2024)
- [OmniNeRF: Hybriding Omnidirectional Distance and Radiance fields for Neural Surface Reconstruction](https://mlsanity.com/paper/2209.13433/) (2022)

## Built on by (most cited first, 24 of 24)

- [3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting](https://mlsanity.com/paper/2412.12507/) (2024)
- [LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Rendering](https://mlsanity.com/paper/2505.23158/) (2025)
- [Gaussian Splatting for Efficient Satellite Image Photogrammetry](https://mlsanity.com/paper/2412.13047/) (2024)
- [Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction](https://mlsanity.com/paper/2502.09563/) (2025)
- [UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations](https://mlsanity.com/paper/2411.15355/) (2024)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [3DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Cameras](https://mlsanity.com/paper/2505.24053/) (2025)
- [ParkGaussian: Surround-view 3D Gaussian Splatting for Autonomous Parking](https://mlsanity.com/paper/2601.01386/) (2026)
- [GFreeDet: Exploiting Gaussian Splatting and Foundation Models for Model-free Unseen Object Detection in the BOP Challenge 2024](https://mlsanity.com/paper/2412.01552/) (2024)
- [SaLF: Sparse Local Fields for Multi-Sensor Rendering in Real-Time](https://mlsanity.com/paper/2507.18713/) (2025)
- [Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI](https://mlsanity.com/paper/2511.20620/) (2025)
- [UniSHARP: Universal Sharp Monocular View Synthesis](https://mlsanity.com/paper/2606.07514/) (2026)
- [FisheyeGaussianLift: BEV Feature Lifting for Surround-View Fisheye Camera Perception](https://mlsanity.com/paper/2511.17210/) (2025)
- [DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization](https://mlsanity.com/paper/2604.00648/) (2026)
- [3D Gaussian Splatting with Fisheye Images: Field of View Analysis and Depth-Based Initialization](https://mlsanity.com/paper/2508.06968/) (2025)
- [Every Camera Effect, Every Time, All at Once: 4D Gaussian Ray Tracing for Physics-based Camera Effect Data Generation](https://mlsanity.com/paper/2509.10759/) (2025)
- [Drive-Through 3D Vehicle Exterior Reconstruction via Dynamic-Scene SfM and Distortion-Aware Gaussian Splatting](https://mlsanity.com/paper/2603.26638/) (2026)
- [Bringing a Personal Point of View: Evaluating Dynamic 3D Gaussian Splatting for Egocentric Scene Reconstruction](https://mlsanity.com/paper/2604.23803/) (2026)
- [UniTriSplat: A Unified 3D Gaussian Splatting Framework with Uniform Spherical Rasterization for Universal Cameras](https://mlsanity.com/paper/2606.29794/) (2026)
- [Differentiable Voronoi Ray Tracing Beyond Rasterization Speeds](https://mlsanity.com/paper/2608.17682/) (2026)
- [ConeGaussian: Anti-Aliased Gaussian Ray-Tracing for Generic Central Cameras](https://mlsanity.com/paper/2609.13397/) (2026)
- [Light Field Primitive for Novel View Synthesis](https://mlsanity.com/paper/2609.31198/) (2026)
- [Feedforward Novel View Synthesis for Heterogeneous Cameras](https://mlsanity.com/paper/2610.03522/) (2026)
- [YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation](https://mlsanity.com/paper/2610.17521/) (2026)

## Abstract (arXiv)

Recently, 3D Gaussian Splatting (3DGS) has garnered attention for its high fidelity and real-time rendering. However, adapting 3DGS to different camera models, particularly fisheye lenses, poses challenges due to the unique 3D to 2D projection calculation. Additionally, there are inefficiencies in the tile-based splatting, especially for the extreme curvature and wide field of view of fisheye lenses, which are crucial for its broader real-life applications. To tackle these challenges, we introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods, emphasizing its extensibility, lightweight nature, and modular design. Since we only modified the projection component, it can also be easily adapted for use with different camera models. Compared to methods that train after undistortion, our approach demonstrates a clear improvement in visual quality.