# FF-ProCams: Feed-Forward Gaussian Splatting for Projector-Camera System

arXiv: https://arxiv.org/abs/2607.17803 · HTML: https://mlsanity.com/paper/2607.17803/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 58 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model](https://mlsanity.com/paper/2311.06214/) (2023)
- [Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation](https://mlsanity.com/paper/2501.12202/) (2025)
- [GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.19702/) (2024)
- [AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views](https://mlsanity.com/paper/2505.23716/) (2025)
- [CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model](https://mlsanity.com/paper/2403.05034/) (2024)
- [DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model](https://mlsanity.com/paper/2311.09217/) (2023)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction](https://mlsanity.com/paper/2311.12024/) (2023)
- [MeshLRM: Large Reconstruction Model for High-Quality Meshes](https://mlsanity.com/paper/2404.12385/) (2024)
- [Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splats](https://mlsanity.com/paper/2410.12781/) (2024)
- [MatSynth: A Modern PBR Materials Dataset](https://mlsanity.com/paper/2401.06056/) (2024)
- [DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading](https://mlsanity.com/paper/2404.09412/) (2024)
- [LRM-Zero: Training Large Reconstruction Models with Synthesized Data](https://mlsanity.com/paper/2406.09371/) (2024)
- [End-to-end Full Projector Compensation](https://mlsanity.com/paper/2008.00965/) (2020)
- [Neural Projection Mapping Using Reflectance Fields](https://mlsanity.com/paper/2306.06595/) (2023)
- [Casper DPM: Cascaded Perceptual Dynamic Projection Mapping onto Hands](https://mlsanity.com/paper/2409.04397/) (2024)
- [Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors](https://mlsanity.com/paper/2403.02547/) (2024)
- [GS-ProCams: Gaussian Splatting-based Projector-Camera Systems](https://mlsanity.com/paper/2412.11762/) (2024)
- [DPCS: Path Tracing-Based Differentiable Projector-Camera Systems](https://mlsanity.com/paper/2503.12174/) (2025)

## Abstract (arXiv)

Projector-camera (ProCams) systems achieve active scene perception and controllable appearance manipulation via structured illumination, serving as a core infrastructure for spatial augmented reality, projection mapping, and surface reflectance acquisition. Existing inverse-rendering methods for ProCams deliver high-fidelity results but rely on time-consuming per-scene optimization, while mainstream feed-forward 3D reconstruction models produce baked appearance that cannot adapt to spatially varying projector illumination. To resolve this accuracy-efficiency trade-off, we propose FF-ProCams, a Feed-Forward 3D Gaussian inverse-rendering framework for ProCams. A hybrid Mamba2-Transformer encoder aggregates cross-view geometric and photometric cues from sparse multi-view observations, and lightweight heads predict a relightable Gaussian representation in a single forward pass. We further design a projector-aware differentiable renderer to synthesize camera observations under arbitrary active illumination and ProCams poses. To enable feed-forward training, we construct a large-scale synthetic ProCams dataset covering diverse object geometries and surface materials. Experiments show FF-ProCams achieves high-fidelity projector-aware rendering, generalizes to unseen patterns, and supports novel projector-camera poses. Using only 8 input views, it outperforms optimization-based baselines with 297 views while reducing test-time reconstruction to 0.13 seconds (a three-to-five-order-of-magnitude speedup). The code and data are available at https://github.com/CPREgroup/FF-ProCams/.