# GS-ProCams: Gaussian Splatting-based Projector-Camera Systems

arXiv: https://arxiv.org/abs/2412.11762 · HTML: https://mlsanity.com/paper/2412.11762/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 96 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 30 of 34)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering](https://mlsanity.com/paper/2311.12775/) (2023)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces](https://mlsanity.com/paper/2311.17977/) (2023)
- [Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes](https://mlsanity.com/paper/2404.10772/) (2024)
- [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance](https://mlsanity.com/paper/2312.00846/) (2023)
- [DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing](https://mlsanity.com/paper/2403.17822/) (2024)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction](https://mlsanity.com/paper/2403.16964/) (2024)
- [DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing](https://mlsanity.com/paper/2312.07409/) (2023)
- [Cross-domain Compositing with Pretrained Diffusion Models](https://mlsanity.com/paper/2302.10167/) (2023)
- [End-to-end Full Projector Compensation](https://mlsanity.com/paper/2008.00965/) (2020)
- [Pro-Cam SSfM: Projector-Camera System for Structure and Spectral Reflectance from Motion](https://mlsanity.com/paper/1908.08185/) (2019)
- [End-to-end Projector Photometric Compensation](https://mlsanity.com/paper/1904.04335/) (2019)
- [Neural Projection Mapping Using Reflectance Fields](https://mlsanity.com/paper/2306.06595/) (2023)
- [Active Stereo Without Pattern Projector](https://mlsanity.com/paper/2309.12315/) (2023)
- [Casper DPM: Cascaded Perceptual Dynamic Projection Mapping onto Hands](https://mlsanity.com/paper/2409.04397/) (2024)

## Built on by (most cited first, 6 of 6)

- [Shadowless Projection Mapping for Tabletop Workspaces with Synthetic Aperture Projector](https://mlsanity.com/paper/2603.11551/) (2026)
- [ProCap: Projection-Aware Captioning for Spatial Augmented Reality](https://mlsanity.com/paper/2604.00912/) (2026)
- [Setup-Independent Full Projector Compensation](https://mlsanity.com/paper/2604.01736/) (2026)
- [Breaking the Scalability Limit of Multi-Projector Calibration with Embedded Cameras](https://mlsanity.com/paper/2604.24024/) (2026)
- [Physically-guided Image Generation for Multi-Projection Mapping](https://mlsanity.com/paper/2606.22477/) (2026)
- [FF-ProCams: Feed-Forward Gaussian Splatting for Projector-Camera System](https://mlsanity.com/paper/2607.17803/) (2026)

## Abstract (arXiv)

We present GS-ProCams, the first Gaussian Splatting-based framework for projector-camera systems (ProCams). GS-ProCams is not only view-agnostic but also significantly enhances the efficiency of projection mapping (PM) that requires establishing geometric and radiometric mappings between the projector and the camera. Previous CNN-based ProCams are constrained to a specific viewpoint, limiting their applicability to novel perspectives. In contrast, NeRF-based ProCams support view-agnostic projection mapping, however, they require an additional co-located light source and demand significant computational and memory resources. To address this issue, we propose GS-ProCams that employs 2D Gaussian for scene representations, and enables efficient view-agnostic ProCams applications. In particular, we explicitly model the complex geometric and photometric mappings of ProCams using projector responses, the projection surface's geometry and materials represented by Gaussians, and the global illumination component. Then, we employ differentiable physically-based rendering to jointly estimate them from captured multi-view projections. Compared to state-of-the-art NeRF-based methods, our GS-ProCams eliminates the need for additional devices, achieving superior ProCams simulation quality. It also uses only 1/10 of the GPU memory for training and is 900 times faster in inference speed. Please refer to our project page for the code and dataset: https://realqingyue.github.io/GS-ProCams/.