# CamP: Camera Preconditioning for Neural Radiance Fields

arXiv: https://arxiv.org/abs/2308.10902 · HTML: https://mlsanity.com/paper/2308.10902/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [RealFusion: 360deg Reconstruction of Any Object from a Single Image](https://mlsanity.com/paper/2302.10663/) (2023)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [Self-Calibrating Neural Radiance Fields](https://mlsanity.com/paper/2108.13826/) (2021)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [SPARF: Neural Radiance Fields from Sparse and Noisy Poses](https://mlsanity.com/paper/2211.11738/) (2022)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [GNeRF: GAN-based Neural Radiance Field without Posed Camera](https://mlsanity.com/paper/2103.15606/) (2021)
- [Neural Rerendering in the Wild](https://mlsanity.com/paper/1904.04290/) (2019)
- [Progressively Optimized Local Radiance Fields for Robust View Synthesis](https://mlsanity.com/paper/2303.13791/) (2023)
- [Optimal whitening and decorrelation](https://mlsanity.com/paper/1512.00809/) (2015)
- [DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fields](https://mlsanity.com/paper/2303.14478/) (2023)
- [SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization](https://mlsanity.com/paper/2104.09125/) (2021)
- [SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction](https://mlsanity.com/paper/2210.04553/) (2022)
- [Why Having 10,000 Parameters in Your Camera Model is Better Than Twelve](https://mlsanity.com/paper/1912.02908/) (2019)

## Built on by (most cited first, 30 of 55)

- [MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos](https://mlsanity.com/paper/2412.04463/) (2024)
- [MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion](https://mlsanity.com/paper/2409.19152/) (2024)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [V3D: Video Diffusion Models are Effective 3D Generators](https://mlsanity.com/paper/2403.06738/) (2024)
- [Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos](https://mlsanity.com/paper/2412.09621/) (2024)
- [Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting](https://mlsanity.com/paper/2402.15870/) (2024)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting](https://mlsanity.com/paper/2403.11831/) (2024)
- [FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent](https://mlsanity.com/paper/2404.15259/) (2024)
- [SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video](https://mlsanity.com/paper/2412.09982/) (2024)
- [Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion](https://mlsanity.com/paper/2403.13327/) (2024)
- [4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video](https://mlsanity.com/paper/2503.18421/) (2025)
- [NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods](https://mlsanity.com/paper/2406.17345/) (2024)
- [A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose](https://mlsanity.com/paper/2405.03659/) (2024)
- [Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solids](https://mlsanity.com/paper/2405.02386/) (2024)
- [SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild](https://mlsanity.com/paper/2401.10171/) (2024)
- [LatentEditor: Text Driven Local Editing of 3D Scenes](https://mlsanity.com/paper/2312.09313/) (2023)
- [Robust Gaussian Splatting](https://mlsanity.com/paper/2404.04211/) (2024)
- [HPC: Hierarchical Progressive Coding Framework for Volumetric Video](https://mlsanity.com/paper/2407.09026/) (2024)
- [Neural Radiance Field-based Visual Rendering: A Comprehensive Review](https://mlsanity.com/paper/2404.00714/) (2024)
- [TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Tracks](https://mlsanity.com/paper/2408.10739/) (2024)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses](https://mlsanity.com/paper/2310.09776/) (2023)
- [Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction](https://mlsanity.com/paper/2502.09563/) (2025)
- [Generic Objects as Pose Probes for Few-shot View Synthesis](https://mlsanity.com/paper/2408.16690/) (2024)
- [FastMap: Revisiting Structure from Motion through First-Order Optimization](https://mlsanity.com/paper/2505.04612/) (2025)
- [Stochastic Preconditioning for Neural Field Optimization](https://mlsanity.com/paper/2505.20473/) (2025)
- [PPISP: Physically-Plausible Compensation and Control of Photometric Variations in Radiance Field Reconstruction](https://mlsanity.com/paper/2601.18336/) (2026)
- [High-Quality Mesh Blendshape Generation from Face Videos via Neural Inverse Rendering](https://mlsanity.com/paper/2401.08398/) (2024)
- [CrossScore: Towards Multi-View Image Evaluation and Scoring](https://mlsanity.com/paper/2404.14409/) (2024)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. However, NeRFs require accurate camera parameters as input -- inaccurate camera parameters result in blurry renderings. Extrinsic and intrinsic camera parameters are usually estimated using Structure-from-Motion (SfM) methods as a pre-processing step to NeRF, but these techniques rarely yield perfect estimates. Thus, prior works have proposed jointly optimizing camera parameters alongside a NeRF, but these methods are prone to local minima in challenging settings. In this work, we analyze how different camera parameterizations affect this joint optimization problem, and observe that standard parameterizations exhibit large differences in magnitude with respect to small perturbations, which can lead to an ill-conditioned optimization problem. We propose using a proxy problem to compute a whitening transform that eliminates the correlation between camera parameters and normalizes their effects, and we propose to use this transform as a preconditioner for the camera parameters during joint optimization. Our preconditioned camera optimization significantly improves reconstruction quality on scenes from the Mip-NeRF 360 dataset: we reduce error rates (RMSE) by 67% compared to state-of-the-art NeRF approaches that do not optimize for cameras like Zip-NeRF, and by 29% relative to state-of-the-art joint optimization approaches using the camera parameterization of SCNeRF. Our approach is easy to implement, does not significantly increase runtime, can be applied to a wide variety of camera parameterizations, and can straightforwardly be incorporated into other NeRF-like models.